Distributed search method in storage device and system using the method

Abstract

The storage system includes a plurality of storage devices in a storage chassis and configured to store objects, an Ethernet switch in the storage chassis, a baseboard management controller (BMC) connected to the storage devices, and a memory connected to the BMC. The plurality of storage devices include a query storage device and a candidate storage device, and the storage devices are connected to each other via the Ethernet switch in the storage chassis. The memory or the query storage device is configured to store metadata corresponding to an object stored in the candidate storage device.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application Serial No. 62 / 634,644, filed on February 23, 2018, entitled “SYSTEM AND METHOD FOR SUPPORTING DISTRIBUTED LOOKUP AND CONTROL MECHANISMS FOR ESSD MACHINE LEARNING QUERIES,” and U.S. Patent Application No. 15 / 961,755, filed on April 24, 2018, the entire contents of which are incorporated herein by reference. Technical Field

[0003] Aspects of example embodiments of the present disclosure relate to a distributed search method in a storage device and a system employing the method. Background Art

[0004] For example, the demand for high-capacity, high-performance storage devices has increased significantly recently due to the increase in file sizes and the increase in the amount of information (or data) being generated and stored. In order to provide larger storage space at a reasonable price, as well as the ability to access data remotely and from a variety of devices, cloud storage has become increasingly popular. Cloud storage (e.g., remote storage) allows users to store and access large amounts of data remotely, giving companies the freedom to acquire additional computing and storage resources as needed without investing a large amount of money in hardware purchases, and also enables individual users to work on more compact devices without being limited by local storage. However, these improvements have placed additional burdens on existing data centers, servers, and data access protocols by increasing the amount of data transferred between data centers and users.

[0005] One of the most energy consuming and therefore costly activities in a data center is the transfer of data to and / or from the data center to external devices, including a user's computer or device, etc. Therefore, it is desirable to reduce or minimize the amount of data transferred to and / or from the data center to reduce energy consumption and costs. Summary of the invention

[0006] The present disclosure relates to various embodiments of a distributed search method in a storage device and a system using the method.

[0007] According to one embodiment of the present disclosure, a storage system includes a plurality of storage devices configured to store objects in a storage chassis, an Ethernet switch in the storage chassis, a BMC (Baseboard Management Controller) connected to the storage devices, and a memory connected to the BMC. The plurality of storage devices include a query storage device and a candidate storage device, and the storage devices are connected to each other via the Ethernet switch in the storage chassis. The memory or the query storage device is configured to store metadata corresponding to an object stored in the candidate storage device.

[0008] The storage devices may include solid-state devices.

[0009] Metadata may include information about one or more corresponding objects.

[0010] The Ethernet switch in the storage enclosure may be configured to communicate with an external Ethernet switch.

[0011] An Ethernet switch in a storage enclosure may be configured to communicate with an Ethernet switch in another storage enclosure.

[0012] The query store may be configured to store metadata, and the query store may be configured to store at least some objects.

[0013] In response to receiving the object store query, the query storage device may be configured to identify which of the storage devices store objects that comply with the object store query.

[0014] The memory may be configured to store metadata, and the query storage device may be configured to store a backup of the metadata stored in the memory.

[0015] In response to receiving the object storage query, the BMC may be configured to identify which of the storage devices store objects that comply with the object storage query.

[0016] According to another embodiment of the present disclosure, a method for performing distributed search by using a storage chassis is provided. The storage chassis includes a query storage device and multiple candidate storage devices, an Ethernet switch connecting the query storage device and the candidate storage devices to each other, a BMC connected to the query storage device and the candidate storage device, and a memory connected to the BMC. The candidate storage device stores multiple objects. The method includes: receiving an object storage query from a host to a query storage device; forwarding the object storage query from the query storage device to the BMC; notifying the query storage device which candidate storage devices among the candidate storage devices store objects that meet the object storage query; and searching the candidate storage devices for objects that meet the object storage query.

[0017] The query storage device and the candidate storage devices may include solid state drives.

[0018] The method may also include: sending objects that match the object store query to the query storage device; and sending objects that match the object store query from the query storage device to the host.

[0019] The memory may store metadata corresponding to objects stored in the candidate storage devices, and the query storage device may instruct the candidate storage devices to perform a search.

[0020] At least one of the candidate storage devices storing objects that match the object storage query may be located in a different storage chassis.

[0021] According to another embodiment of the present disclosure, a method for performing distributed search by using a storage chassis is provided. The storage chassis includes a query storage device and multiple candidate storage devices storing multiple objects, an Ethernet switch connecting the query storage device and the candidate storage devices to each other, and a BMC connected to the query storage device and the candidate storage devices. The method includes: receiving an object storage query from a host to the query storage device; determining which of the candidate storage devices store objects that meet the object storage query; and searching for candidate storage devices that store objects that meet the object.

[0022] The method may also include sending the conforming object to the query storage device via the Ethernet switch.

[0023] The query storage device and the candidate storage devices may include solid state drives.

[0024] The query storage device may store metadata corresponding to the object.

[0025] Determining which of the candidate storage devices store objects that comply with the object storage query may include determining that a storage device in another storage enclosure stores objects that comply with the object storage query.

[0026] The method may also include sending the operational information from the querying storage device and the candidate storage device to the BMC via the PCIe switch.

[0027] The querying storage device may communicate with another storage device in another storage enclosure via an Ethernet switch.

[0028] This summary is provided to introduce a selection of features and concepts of example embodiments of the present disclosure, which are further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. One or more described features according to one or more example embodiments can be combined with one or more other described features according to one or more example embodiments to provide a feasible method or apparatus. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A data storage environment according to an embodiment of the present disclosure is shown;

[0030] Figure 2 Shown in Figure 1 The distributed search method used in the data storage environment shown;

[0031] Figure 3 shows a data storage environment according to another embodiment of the present disclosure; and

[0032] Figure 4 Shown in Figure 3 The distributed search method used in the data storage environment shown. DETAILED DESCRIPTION

[0033] The present disclosure relates to various embodiments of a distributed search method in a storage device and a system using the method. According to an embodiment of the present disclosure, a storage chassis includes multiple storage devices and a BMC (baseboard management controller). One or more of the storage devices and / or the BMC may store metadata and / or feature information of data (e.g., objects) stored in the storage devices in the storage chassis. Therefore, when a user submits an object storage query via a host (e.g., a remote host) or a user device, a storage device that may have data that meets the object storage request can be determined based on the metadata and / or feature information stored in the storage chassis, and the compliant objects can be organized and transmitted from the storage chassis to the user (e.g., to the host or user device) as a collective object query response.

[0034] Hereinafter, example embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. However, the present disclosure may be embodied in various different forms and should not be construed as being limited to the embodiments shown herein. On the contrary, these embodiments are provided as examples so that the present disclosure will be comprehensive and complete, and the various aspects and features of the present disclosure will be fully conveyed to those skilled in the art. Therefore, in order to fully understand the aspects and features of the present disclosure, processes, elements, and techniques that are unnecessary for those of ordinary skill in the art may not be described. Unless otherwise stated, the same reference numerals represent the same elements throughout the drawings and written descriptions, and therefore, their descriptions may not be repeated.

[0035] It should be understood that although the terms "first", "second", "third", etc. may be used herein to describe various elements, components and / or layers, these elements, components and / or layers should not be limited by these terms. These terms are used to distinguish one element, component or layer from another element, component or layer. Therefore, without departing from the scope of the present disclosure, the first element, component or layer described below may be referred to as a second element, component or layer.

[0036] It should be understood that when an element or component is referred to as being "connected to" or "coupled to" another element or component, it may be directly connected or coupled to the other element or component, or one or more intermediate elements or components may be present. When an element or component is referred to as being "directly connected to" or "directly coupled to" another element or component, there are no intermediate elements or components. For example, when a first element is described as being "coupled to" or "connected to" a second element, the first element may be directly coupled or connected to the second element, or the first element may be indirectly coupled or connected to the second element via one or more intermediate elements.

[0037] The terms used herein are for the purpose of describing specific embodiments, rather than intending to limit the present disclosure. As used herein, unless the context clearly states otherwise, the singular forms "one" and "an" are also intended to include plural forms. It should be further understood that when used in this specification, the terms "include", "include", "include" and "include" specify the presence of the features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. That is, the processes, methods and algorithms described herein are not limited to the indicated operations, and may include additional operations or may omit some operations, and the order of operations may vary according to some embodiments. As used herein, the term "and / or" includes any and all combinations of one or more associated listed items.

[0038] As used herein, the terms "substantially," "approximately," and similar terms are used as approximations rather than terms of degree, and are intended to account for the inherent variations in measurements or calculations that would be recognized by one of ordinary skill in the art. In addition, when describing embodiments of the present disclosure, the usage of "may" refers to "one or more embodiments of the present disclosure." As used herein, the terms "use," "using," and "used" may be considered synonyms of "utilizing," "utilizing," and "utilized," respectively. Moreover, the term "example" is intended to represent an example or illustration.

[0039] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those of ordinary skill in the art to which the present disclosure belongs. It should be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having the same meaning as they have in the context of the technology of the relevant art and / or this specification, and should not be interpreted as idealized or overly formal unless explicitly defined herein.

[0040] According to an embodiment of the present disclosure, the processor, central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), hard disk drive (HDD), solid-state drive (SSD) and / or any other related device or component described herein can be implemented using any suitable hardware (e.g., dedicated integrated circuit), firmware, software and / or a suitable combination of software, firmware and hardware. For example, various components of the processor, CPU, GPU, FPGA, HDD and / or SSD can be formed (or implemented) on an integrated circuit (IC) chip or formed on a single IC chip. In addition, various components of the processor, CPU, GPU, FPGA, HDD and / or SSD can be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on the same substrate as the processor, CPU, GPU, FPGA, HDD and / or SSD. In addition, the actions described may be processes or threads running on one or more processors (e.g., one or more CPUs, GPUs, FPGAs, etc.) in one or more computing devices, executing computer program instructions and interacting with other system components to perform the various functions described herein. Computer program instructions may be stored in a memory that may be implemented in a computing device using a standard memory device such as, for example, a random access memory (RAM). Computer program instructions may also be stored in other non-temporary computer-readable media such as, for example, a CD-ROM, a flash drive, a HDD, an SSD, etc. Furthermore, those skilled in the art should recognize that the functionality of various computing devices may be combined or integrated into a single computing device, or that the functionality of a particular computing device may be distributed across one or more other computing devices without departing from the scope of the exemplary embodiments of the present disclosure.

[0041] refer to Figure 1 , shows a data storage environment 1000 according to an embodiment of the present disclosure. The data storage environment 1000 includes an external server 100 and a storage repository 200. The data storage environment 1000 may utilize an object storage (e.g., object-based storage) architecture, and an embodiment adopting the object storage architecture will be described below, but the present disclosure is not limited thereto.

[0042] In an object storage architecture, an object server, discussed further below, allows a client (e.g., a remote host or user device) to store and retrieve data (e.g., data objects or objects) residing in (e.g., stored in) a data repository (e.g., an object repository). The data repository may include one or more storage enclosures, and each of the storage enclosures includes a plurality of storage devices on which objects are stored.

[0043] In one embodiment, the external server 100 includes a plurality of object servers 1, 2, ..., N represented by 101-103, a plurality of query servers 1, ..., M represented by 104 and 105, and an Ethernet switch 110. Although the plurality of object servers 101-103 and the plurality of query servers 104, 105 are shown as part of the external server 100, the object servers and the query servers may be services running on one or more computers (or servers, etc.) and may not be physically independent servers. In addition, a single computer (or server) may alternate between acting as an object server and a query server, the object server providing data to the repository 200 for storage, and the query server querying the repository 200 to retrieve the requested data. In some embodiments, a single computer (or server) may act as both an object server and a query server in parallel or simultaneously.

[0044] Each of the object servers 101-103 and the query servers 104, 105 is connected to the storage repository 200 via an Ethernet switch 110. However, the present disclosure is not limited to connections based on Ethernet interfaces, and other suitable connection interfaces may also be used. Figure 1 , the Ethernet switch 110 is shown as being connected to the storage library 200 via three Ethernet cables. However, the present disclosure is not limited thereto, and any suitable number of Ethernet cables may be used to connect the Ethernet switch 110 to the storage library 200, with an increased number of Ethernet cables providing increased bandwidth between the Ethernet switch 110 and the storage library 200.

[0045] The storage library 200 includes one or more storage chassis 200.1. The storage chassis 200.1 includes a plurality of storage devices 1, 2, ..., K represented by 201-203, a chassis Ethernet switch 210, and a chassis controller including a PCIe (Peripheral Component Interconnect Express) switch 220, a BMC (Baseboard Management Controller) 221, memories 222.1 and 222.2, and a chassis CPU 223. In one embodiment, the storage devices 201-203 may be solid state drives (SSDs). The memories 222.1 and 222.2 may be DRAMs (Dynamic Random Access Memory). The BMC memory 222.1 may be dedicated to the BMC 221, and the CPU memory 222.2 may be dedicated to the chassis CPU 223. In some embodiments, the chassis CPU 223 may be omitted, wherein the BMC 221 performs control functions for the storage devices and for the chassis Ethernet switch 210.

[0046] In some embodiments, storage devices 201-203 may be Ethernet-connected SSDs (eSSDs) that receive and send data using an Ethernet interface. Hereinafter, an embodiment in which storage devices 201-203 are Ethernet-connected SSDs will be described, but the present disclosure is not limited thereto.

[0047] The storage repository 200 may include another (or second) storage chassis connected to the first storage chassis 200.1 via an Ethernet connection 211. In this manner, multiple storage chassis may be connected to each other (e.g., daisy-chained together) to increase the amount of storage available in the storage repository 200 and / or to provide redundancy for data stored in the storage repository in the event of a device failure, power failure, etc. In addition, the Ethernet switch 110 may also be connected to the second storage chassis via an Ethernet connection to increase transfer rates, etc. In some embodiments, for example, the second storage chassis may be located relatively far away from the first storage chassis 200.1, and the first and second storage chassis may be connected to each other via the Internet. However, as described above, in some embodiments, the storage repository 200 may include only a single storage chassis 200.1.

[0048] In the storage chassis 200.1, each of the storage devices 201-203 is connected to the chassis Ethernet switch 210 via an Ethernet connection, and is also connected to the PCIe switch 220 via a PCIe connection. The storage devices 201-203 can communicate with each other via the chassis Ethernet switch 210 and / or the PCIe switch 220. As some examples, the PCIe connection can be used as a control plane or connection (e.g., a management interface) that allows the storage devices 201-203 to report various states to the BMC 221 for monitoring and reporting. The PCIe connection is not typically used to send or receive stored data. By using the PCIe switch 220 to communicate with each other via the control plane, the storage devices 201-203 can communicate with each other without utilizing the bandwidth available on the Ethernet interface or data plane to transfer data into or out of the storage chassis 200.1.

[0049] In some embodiments, storage devices 201-203 can manage their own internal storage. For example, when one of storage devices 201-203 runs out of its storage capacity, it can request overflow storage on another of storage devices 201-203 or on a different storage device in another storage chassis. In order to request overflow storage, the storage device can contact BMC 221 via PCIe interface, and BMC 221, which maintains information about the remaining storage capacity of storage devices 201-203, can allocate a portion of another of the storage devices to the exhausted storage device as overflow storage. In some cases, BMC 221 can maintain information about the remaining storage capacity of storage devices in other storage chassis via Ethernet connection 211, and can allocate a portion of another storage device to a different storage chassis as overflow storage.

[0050] In an object storage architecture, data is stored as objects, and each object has corresponding metadata and feature information. Metadata may include information about the corresponding object. For example, when the object is a picture file, the metadata may describe the subject of the picture file, such as whether the picture is a picture of a dog. In recent years, machine learning algorithms have been adopted to automatically process, classify and / or organize data in repositories, and information collected from objects may be stored in metadata through such machine learning algorithms. For example, feature information may include the file type of the object, such as JPEG, GIF, MP3, etc.

[0051] Conventionally, object metadata and feature information are stored in query servers separate from the repository, and each query server maintains a database storing certain objects. For example, the query server stores data indicating which storage devices and / or storage chassis the object is stored on. When a host generates an object storage query (e.g., a retrieval request), the query server interrogates various storage chassis and storage devices in the storage chassis to locate and retrieve objects that conform to the object storage query. As some examples, the object storage query may reference a specific metadata field (e.g., a picture of a dog) or feature information (e.g., a JPEG file type) and / or may include example objects with specified object conformance (e.g., a request to retrieve pictures of dogs of other dogs). In response to the object storage query, the query server communicates with various different storage chassis or storage devices based on the location of the conforming objects (e.g., objects that conform to the object storage query), and the query server collects the conforming objects as object responses and passes them along the way to the host. Transmitting objects from the repository to the requesting query server is energy intensive and uses bandwidth between the query server and the repository.

[0052] According to the first embodiment, the storage devices 201-203 in the storage library 200 report metadata and feature information of the stored objects (e.g., objects stored in one or more of the storage devices 201-203) to the BMC 221 via the chassis Ethernet switch 210 and / or the PCIe switch 220. The BMC 221 may store the feature information and / or metadata of the objects on the storage devices 201-203 in the corresponding storage chassis 200.1. For example, the BMC 221 may store information indicating which of the storage devices 201-203 are storing certain types of files, etc. This information may be stored by the BMC 221 on the BMC memory 222.1; however, in other embodiments, the BMC 221 may store this information in one or more of the storage devices 201-203. In some embodiments, the BMC 221 may store this information in the BMC memory 222.1, but may also store a backup of this information in one or more of the storage devices 201-203 to ensure that the information is not lost during, for example, a power outage or device failure. In addition, storage devices 201 - 203 may report object deletions to BMC 221 so that unnecessary lookups based on deleted objects may be avoided, as will be further described below.

[0053] In other embodiments, the chassis CPU 223 may be used to store characteristic information and / or metadata of stored objects. However, because the BMC 221 may already be tracking the status of the connected storage devices 201-203, including their on / off status, capacity, remaining capacity, etc., storing the information on the BMC 221 may reduce the communication between the BMC 221 and the chassis CPU 223 and provide faster response time for object storage queries.

[0054] Reference Figure 1 and Figure 2 According to an embodiment, a host generates an object storage query (e.g., a retrieval request), and the object storage query is directed to and received by one or more of the storage devices 201-203 in the storage repository 200 via an Ethernet interface (S100). In some embodiments, the object storage query may be directed to storage devices in multiple storage chassis of the storage repository 200 via an Ethernet interface. As described above, the object storage query may refer to object metadata, feature information, may be an example result candidate, etc. In this example, the storage device 201 will be regarded as a receiving storage device, but the present disclosure is not limited thereto.

[0055] Then, the receiving storage device 201 forwards the received object storage query to the corresponding BMC 221 via the PCIe switch 220 (S105). In other embodiments, the receiving storage device 201 may forward the received object storage query to the corresponding BMC 221 via the chassis Ethernet switch 210, or may select which of the chassis Ethernet switch 210 or the PCIe switch 220 to use based on bandwidth availability. After the receiving storage device 201 reports the received object storage query to the BMC 221, it may be referred to as a reporting storage device.

[0056] Then, BMC 221 notifies receiving (or reporting) storage device 201 of various storage devices (e.g., any storage device 201-203 in storage chassis 200.1 or external storage devices in other storage chassis of storage repository 200) that can store the compliant object in response to the object storage query (S110). Storage devices that can store the compliant object in response to the object storage query can be referred to as candidate storage devices. In this example, storage devices 202, 203 will be considered as candidate storage devices, but the present disclosure is not limited thereto. In some cases, receiving (or reporting) storage device 201 can also be a candidate storage device.

[0057] Then, the reporting storage device 201 instructs the candidate storage devices 202, 203 to perform a search based on the object storage query (S115). When external storage devices in other storage chassis are identified as candidate storage devices by the BMC 221, the reporting storage device 201 can instruct those external candidate storage devices to perform a search to another storage chassis via the Ethernet connection 211. Then, the candidate storage devices 202, 203 report any objects that meet the object storage query to the reporting storage device 201 (S120). The reporting can occur on the chassis Ethernet switch 210 and / or PCIe switch 220 in the storage chassis 200.1.

[0058] After the reporting storage device 201 receives communications from each of the candidate storage devices 202, 203 including a matching object (i.e., an object that matches the object storage query) or a message indicating that no matching object was found, for example, when a previously matching object was deleted and the deletion was not reported to the BMC 221, the reporting storage device 201 returns the matching object to the host as a response to the object storage query (S125).

[0059] In other embodiments, instead of BMC 221 storing metadata and feature information of objects stored in all storage devices 201-203 or in addition to BMC 221 storing such information (e.g., BMC 221 combined with BMC memory 222.1), each of storage devices 201-203 may store metadata and feature information of objects stored in all storage devices 201-203. Because the metadata and feature information of an object is much smaller than the object itself, the storage penalty (penalty) of each storage device 201-203 storing metadata and feature information of objects stored in all storage devices 201-203 is relatively small. In this embodiment, communication between the receiving (or reporting) storage device 201 and BMC 221 may be omitted because the receiving storage device 201 determines which of the storage devices 201-203 are candidate storage devices based on the object storage query. By omitting communication with BMC 221, the response time to the object storage query may be shortened.

[0060] like Figure 1As shown, when the storage repository 200 is used with the query servers 104, 105, the query servers 104, 105 can pass through the object storage query from the host to the storage chassis 200.1. For example, unlike the above-mentioned traditional data storage environment, in an embodiment of the present disclosure, object metadata and feature information are stored locally on the storage chassis 200.1. In this way, the amount of information sent to and from the storage chassis 200.1 is reduced, and the query speed is improved. When the storage chassis 200.1 is connected to a traditional query server, the traditional query server can pass along the object storage query to the storage chassis 200.1 without querying each storage device as in the traditional data storage environment. In this way, the storage chassis 200.1 according to an embodiment of the present disclosure can be used with a traditional query server, thereby simplifying the conversion of companies and network administrators.

[0061] refer to Figure 3 According to another embodiment, a data storage environment 2000 includes a storage repository 250 and a remote host 300. The storage repository 250 includes one or more storage chassis 250.1. The storage chassis 250.1 includes a plurality of storage devices 1, 2, ..., K represented by 251-253 and a chassis Ethernet switch 260. Each of the storage devices 251-253 is connected to the chassis Ethernet switch 260, and as described above, may be a solid state drive (SSD), and in some embodiments, may be an Ethernet SSD (eSSD). Similar to Figure 1 The storage chassis 200.1 shown in the figure, the storage chassis 250.1 also includes a PCIe switch 220 connected to each of the storage devices 251-253, a BMC 221 connected to the PCIe switch 220, and a memory 222 connected to the BMC 221. For ease of description, the PCIe switch 220, the BMC 221, and the memory 222 may not be described again, which is basically similar to the above description about Figure 1 The BMC memory 222.1 of the embodiment shown.

[0062] Storage Chassis 250.1 omitted Figure 1 2. However, as described above with reference to storage chassis 200.1, chassis CPU 223 may optionally be included in storage chassis 250.1, or may be omitted.

[0063] The storage enclosure 250.1 is connected to a remote host (eg, a host or user device) 300 via a network connection 350. The network connection 350 may be, for example, the Internet, a local area network, etc. Figure 1Unlike the illustrated embodiment, the storage enclosure 250.1 is connected to the remote host 300 without imposing a query server. However, the storage enclosure 250.1 may be connected to one or more query servers, such as Figure 1 Similarly, Figure 1 The storage enclosure 200 . 1 shown in FIG. 1 may be connected to a remote host 300 without imposing a query on the servers 104 , 105 .

[0064] refer to Figure 3 and Figure 4 According to another embodiment, the remote host 300 generates an object storage query and sends the object storage query to the storage enclosure 250.1 via the network connection 350. Then, the storage enclosure 250.1 receives the object storage query (S200).

[0065] The object storage query is then directed to one or more of the storage devices 251-253 acting as a query server. For example, one of the storage devices 251-253 may act as a query server by storing metadata and feature information of all objects stored in the storage devices 251-253. Figure 1 The storage chassis 200.1 shown is different, in that the BMC 221 stores metadata and feature information of the stored objects. Figure 3 In the storage enclosure 250.1 shown, one of the storage devices 251-253 acts as a query server. In this example, the storage device 251 will be considered as a query server (or query storage device), but the present disclosure is not limited thereto.

[0066] When the storage device 251 is a query server (or query storage device), the object storage query is directed to the storage device 251 through the chassis Ethernet switch 260. Then, the storage device 251 determines which of the storage devices 251-253 are candidate storage devices based on the object storage query (S205). In this example, the storage devices 252, 253 will be regarded as candidate storage devices, but the present disclosure is not limited to this. For example, in some cases, the storage device (query storage device) 251 can also store objects. That is, the query storage device 251 can also act as a storage device for storing objects. Because the size of the metadata and feature information of the object can be substantially smaller than the size of the object itself, the query storage device 251 can have a relatively large amount of unused storage space or capacity. In some embodiments, when other storage devices 252, 253 in the storage chassis 250.1 become full, the query storage device 251 can act as overflow storage allocated by the BMC 221.

[0067] The storage device 251 then instructs the candidate storage devices 252, 253 to perform a search therein for objects that conform to the object storage query (S210). The candidate storage devices 252, 253 then report conforming objects found during the search or a message indicating that no conforming objects were found during the search (S215). The reporting may occur via the chassis Ethernet switch 260 and / or the PCIe switch 220. After all candidate storage devices 252, 253 have reported, the storage device 251 returns the conforming objects as a collective object response to the remote host 300 (S220).

[0068] In this embodiment, because one of the storage devices 251-253 acts as a query server, object storage queries can be processed faster because the step of communicating with the BMC 221 to complete the object storage query can be omitted. In addition, a traditional query server outside (e.g., remote from) the storage chassis 250.1 can also be omitted, thereby simplifying the data storage environment 2000.

[0069] Additionally, recent developments have provided machine learning capabilities in storage devices such as SSDs. Thus, data can now be processed and classified by using machine learning algorithms on the storage device itself, rather than sending the data to a computer, server, etc. that is remote from the storage device for processing and classification. By performing the processing and classification on the storage device, the amount of data sent to and from the storage device is reduced. The addition of features of embodiments of the present disclosure further reduces the amount of data sent to and from the storage chassis by storing metadata for objects on a local device such as the storage device itself or a BMC (e.g., on a device in a storage chassis).

[0070] Although the present disclosure has been described with reference to example embodiments, those skilled in the art will recognize that various changes and modifications may be made to the described embodiments without departing from the spirit and scope of the present disclosure. In addition, those skilled in the art will recognize that the present disclosure described herein will suggest solutions to other tasks and adaptations to other applications. It is the applicant's intention to cover all of these uses of the present disclosure by the claims herein, as well as those changes and modifications that may be made to the exemplary embodiments of the present disclosure selected for the purposes of disclosure, all of which do not depart from the spirit and scope of the present disclosure. Therefore, the example embodiments of the present disclosure should be considered in all respects to be illustrative and not restrictive, and the spirit and scope of the present invention are indicated by the appended claims and their equivalents.

Claims

1. A storage system for distributed search in a storage device, comprising: a plurality of storage devices in a storage chassis and configured as storage objects, the plurality of storage devices including a query storage device and a candidate storage device; An Ethernet switch in the storage chassis, wherein the storage devices are connected to each other via the Ethernet switch in the storage chassis; A baseboard management controller BMC connected to the storage device; as well as memory, which is connected to the BMC, wherein the memory or the query storage device is configured to store metadata corresponding to the object stored in the candidate storage device; The storage device receives, from a host, an object storage query including a retrieval request for the object; as well as The query storage device is configured to instruct the candidate storage device to perform a search for objects that comply with the object storage query based on the object storage query.

2. The storage system according to claim 1, wherein: The storage device comprises a solid state device.

3. The storage system according to claim 2, wherein: The metadata includes information about one or more corresponding objects.

4. The storage system according to claim 2, wherein: The Ethernet switch in the storage enclosure is configured to communicate with an external Ethernet switch.

5. The storage system according to claim 4, wherein: The Ethernet switch in the storage enclosure is configured to communicate with an Ethernet switch in another storage enclosure.

6. The storage system according to claim 2, wherein: The query storage device is configured to store the metadata, and Wherein, the query storage device is configured to store at least some of the objects.

7. The storage system according to claim 6, wherein: In response to receiving an object store query, the query storage device is configured to identify which of the storage devices store objects that comply with the object store query.

8. The storage system according to claim 2, wherein: The memory is configured to store the metadata, and Wherein, the query storage device is configured to store a backup of the metadata stored in the memory.

9. The storage system according to claim 8, wherein: In response to receiving an object storage query, the BMC is configured to identify which of the storage devices store objects that comply with the object storage query.

10. A method for performing distributed search by using a storage chassis, the storage chassis comprising: querying a storage device and a plurality of candidate storage devices, wherein the candidate storage devices store a plurality of objects; An Ethernet switch interconnecting the query storage device and the candidate storage device; A baseboard management controller BMC connected to the query storage device and the candidate storage device; and a memory connected to the BMC, the method comprising: receiving, from a host to the query storage device, an object storage query including a retrieval request for the object; forwarding the object storage query from the query storage device to the BMC; Notifying the query storage device which of the candidate storage devices store objects that match the object storage query; The query storage device instructs the candidate storage device to perform a search based on the object storage query, and The candidate storage devices are searched for objects that meet the object storage query.

11. The method of claim 10, wherein: The query storage device and the candidate storage devices include solid state drives.

12. The method of claim 11, further comprising: sending objects matching the object store query to the query store; as well as Objects that match the object store query are sent from the query store to the host.

13. The method of claim 11, wherein: The memory stores metadata corresponding to the object stored in the candidate storage device.

14. A method for performing distributed search by using a storage chassis, the storage chassis comprising: querying a storage device and a plurality of candidate storage devices storing a plurality of objects; An Ethernet switch interconnecting the query storage device and the candidate storage device; and a baseboard management controller BMC connected to the query storage device and the candidate storage device, the method comprising: receiving, from a host to the query storage device, an object storage query including a retrieval request for the object; determining which of the candidate storage devices store objects that match the object storage query; Searching for candidate storage devices storing the matching object; and The query storage device instructs the candidate storage device to perform a search for the matching object based on the object storage query.

15. The method of claim 14, further comprising sending the conforming objects to the query storage device via the Ethernet switch.

16. The method of claim 15, wherein: The query storage device and the candidate storage devices include solid state drives.

17. The method of claim 16, wherein: The query storage device stores metadata corresponding to the object.

18. The method of claim 16, wherein: Determining which of the candidate storage devices store objects that comply with the object storage query includes determining that a storage device in another storage enclosure stores objects that comply with the object storage query.

19. The method of claim 16, further comprising sending operation information from the query storage device and the candidate storage device to the BMC via a PCIe switch.

20. The method of claim 16, wherein: The query storage device communicates with another storage device in another storage chassis via the Ethernet switch.

Images