Search method, host and near memory processing device

Abstract

The invention provides a search method, a host and near memory processing equipment. A search method performed by at least one processor of a host, the method comprising: invoking a near-memory processing (PNM) device to perform search-related computations on a vector data set in a vector database; and receiving a result associated with the search related calculation from the PNM device. A search method performed by at least one processor of a near-memory processing (PNM) device, the method comprising: in response to receiving a call indication from a host, performing a search-related computation on a vector data set in a vector database; and returning a result associated with the search-related calculation to the host after the search-related calculation is completed.

Description

Technical Field

[0001] The present disclosure relates to the field of storage, and in particular to a search method, a host, and a near memory processing device. Background Art

[0002] Vector databases store representations of unstructured data (such as images, text, video, or audio) (e.g., high-dimensional vectors, or simply vectors). Commonly used vector similarity algorithms are also integrated or implemented within vector databases. Based on vector similarity search, vector databases can identify the most similar data items within a vector dataset. This can be used in recommendation systems (including recommending similar products in online shopping, finding similar images, or recommending similar videos on video sites), identifying biological protein structures or gene sequences, detecting financial fraud, and analyzing IoT data.

[0003] Current vector similarity search computations based on the central processing unit (CPU) frequently and repeatedly load vectors from memory and write back vector calculation results. This large number of vector similarity calculations increases the CPU workload. Furthermore, the increasing size of vector datasets has become a performance bottleneck for vector similarity search. Consequently, large amounts of data are moved between the CPU and memory while maintaining a relatively narrow data bandwidth between them. Furthermore, the CPU is burdened with a large number of vector similarity calculation tasks, impacting the overall system response time. Summary of the Invention

[0004] The present disclosure provides a search method, a host, and a near memory processing device to solve part or all of the above problems.

[0005] According to one aspect of the present disclosure, a search method performed by at least one processor of a host is provided. The method may include: invoking a near memory processing (PNM) device to perform search-related calculations on a vector data set in a vector database; and receiving results associated with the search-related calculations from the PNM device.

[0006] Optionally, calling the PNM device is based on sending a calling indication to the PNM device through an interface provided by the vector database.

[0007] Optionally, the calling of the PNM device includes one of the following: providing index parameters to the PNM device through the index establishment interface of the vector database, and calling the PNM device to establish one or more indexes for the vectors in the vector data set, wherein the one or more indexes are used to perform searches on the vector data set, and the index parameters include: index type; or providing search parameters to the PNM device through the search interface of the vector database, and calling the PNM device to search for a query vector in the vector data set, wherein the search parameters include: query vector and vector data set storage location.

[0008] Optionally, the calling PNM device establishes one or more indexes for the vectors in the vector dataset, including: based on the index type having a value associated with the PNM device, calling the PNM device to establish the one or more indexes for the vectors in the vector dataset, and wherein, the calling PNM device searches for a query vector in the vector dataset, including: based on the storage location of the vector dataset in a storage device integrated with the PNM device, calling the PNM device to search for a query vector in the vector dataset.

[0009] Optionally, receiving results associated with search-related calculations from the PNM device includes: based on the PNM device being called to establish the one or more indexes for the vectors in the vector data set, receiving the results of completion of establishing the one or more indexes from the PNM device; or based on the PNM device being called to search for a query vector in the vector data set, receiving search results associated with the query vector from the PNM device.

[0010] According to one aspect of the present disclosure, a search method performed by at least one processor of a near memory processing (PNM) device is provided. The method may include: in response to receiving a call indication from a host, performing a search-related calculation on a vector data set in a vector database; and after the search-related calculation is completed, returning a result associated with the search-related calculation to the host.

[0011] Optionally, the search-related calculation is performed in response to a call instruction sent by the host via an interface provided by the vector database.

[0012] Optionally, performing search-related calculations includes one of the following: establishing one or more indexes for vectors in the vector dataset based on index parameters provided by the host using an index establishment interface of the vector database, wherein the one or more indexes are used to perform a search on the vector dataset, and the index parameters include: an index type; or searching for a query vector in the vector dataset based on search parameters provided by the host using a search interface of the vector database, wherein the search parameters include: a query vector and a vector dataset storage location.

[0013] Optionally, establishing one or more indexes for the vectors in the vector dataset includes: loading the vector dataset from a storage device integrated with the PNM device into a device memory of the PNM device, establishing the one or more indexes for the vectors in the vector dataset loaded into the device memory of the PNM device based on a similarity search method indicated by an index type, and writing the one or more indexes from the device memory of the PNM device to the storage device integrated with the PNM device.

[0014] Optionally, the search parameters also include: a similarity search method, and wherein, based on the search parameters, searching for a query vector in the vector dataset includes: based on a storage location of the vector dataset, loading the vector dataset from a storage device integrated with the PNM device into a device memory of the PNM device, and based on the similarity search method, searching for multiple vectors having similarity to the query vector in the vector dataset loaded into the device memory of the PNM device.

[0015] Optionally, results associated with search-related calculations are returned to the host, including: based on the PNM device being called to establish the one or more indexes for the vectors in the vector data set, returning the results of the completion of establishing the one or more indexes to the host; or based on the PNM device being called to search for the query vector in the vector data set, returning the search results associated with the query vector to the host.

[0016] According to one aspect of the present disclosure, a host is provided, comprising: a memory storing instructions, and at least one processor configured to execute the instructions. The at least one processor is configured to execute the instructions to perform the following operations: invoking a near memory processing (PNM) device to perform search-related calculations on a vector data set in a vector database; and receiving results associated with the search-related calculations returned by the PNM device.

[0017] Optionally, the at least one processor is further configured to execute the instructions to perform the following operations: send a call indication to the PNM device through an interface provided by the vector database, and call the PNM device to perform search-related calculations on the vector data set.

[0018] Optionally, the at least one processor is further configured to execute the instructions to perform the following operations: providing index parameters to the PNM device through the index establishment interface of the vector database, calling the PNM device to establish one or more indexes for the vectors in the vector data set, wherein the one or more indexes are used to perform a search on the vector data set, and the index parameters include: index type; or providing search parameters to the PNM device through the search interface of the vector database, calling the PNM device to search for a query vector in the vector data set, wherein the search parameters include: query vector and vector data set storage location.

[0019] Optionally, the at least one processor is further configured to execute the instructions to perform the following operations: based on the index type having a value associated with the PNM device, calling the PNM device to establish the one or more indexes for the vectors in the vector dataset; and based on the storage location of the vector dataset in a storage device integrated with the PNM device, calling the PNM device to search for a query vector in the vector dataset.

[0020] Optionally, the at least one processor is further configured to execute the instructions to perform the following operations: based on the PNM device being called to establish the one or more indexes for the vectors in the vector data set, receiving the results of the completion of establishing the one or more indexes from the PNM device; or based on the device being called to search for a query vector in the vector data set, receiving search results associated with the query vector from the PNM device.

[0021] According to one aspect of the present disclosure, a near memory processing (PNM) device is provided, comprising a device memory and a computing unit comprising at least one processor. The at least one processor is configured to: in response to receiving a call indication from a host, perform a search-related computation on a vector dataset in a vector database; and after the search-related computation is completed, return a result associated with the search-related computation to the host.

[0022] Optionally, the at least one processor is further configured to: perform search-related calculations on the vector data set in response to a call indication sent by the host via an interface provided by the vector database.

[0023] Optionally, the at least one processor is further configured to: establish one or more indexes for the vectors in the vector dataset based on index parameters provided by the host using an index establishment interface of the vector database, wherein the one or more indexes are used to perform a search on the vector dataset, and wherein the index parameters include: an index type; or search for a query vector in the vector dataset based on search parameters provided by the host using a search interface of the vector database, wherein the search parameters include: a query vector and a vector dataset storage location.

[0024] Optionally, the at least one processor is further configured to: load the vector dataset from a storage device integrated with the PNM device into the device memory, establish the one or more indexes for the vectors in the vector dataset loaded into the device memory based on the similarity search method indicated by the index type, and write the one or more indexes from the device memory to the storage device integrated with the PNM device.

[0025] Optionally, the search parameters also include: a similarity search method, and wherein the at least one processor is further configured to: load the vector dataset from a storage device integrated with the PNM device into the device memory based on the vector dataset storage location, and search for multiple vectors having similarity with the query vector in the vector dataset loaded into the device memory based on the similarity search method.

[0026] Optionally, the at least one processor is further configured to: based on the PNM device being called to establish the one or more indexes for the vectors in the vector data set, return the result of completing the establishment of the one or more indexes to the host; or based on the device being called to search for the query vector in the vector data set, return the search results associated with the query vector to the host.

[0027] According to one aspect of the present disclosure, a system using a near memory processing (PNM) device is provided, including: a main processor; a memory; a PNM device; and a storage device, wherein the PNM device is configured to execute the search method described above.

[0028] According to one aspect of the present disclosure, a host storage system is provided, comprising a host; and a near memory processing (PNM) device. The host may be configured to call the near memory processing (PNM) device to perform search-related computations on a vector dataset in a vector database; and receive results associated with the search-related computations returned by the PNM device. The PNM device may be configured to, in response to receiving a call indication from the host, perform search-related computations on the vector dataset in the vector database; and, after completion of the search-related computations, return the results associated with the search-related computations to the host.

[0029] According to one aspect of the present disclosure, a data center system is provided, comprising: a plurality of application servers; and a plurality of storage servers, wherein each storage server comprises a near memory processing (PNM) device, wherein the PNM device is configured to execute the search method described above.

[0030] According to one aspect of the present disclosure, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the search method as described above is implemented.

[0031] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0033] Figure 1 A conceptual diagram showing a vector database.

[0034] Figure 2 A workflow diagram showing access to a vector database.

[0035] Figure 3 A schematic diagram of the CPU-based vector similarity search process is shown.

[0036] Figure 4A and Figure 4B Shows a schematic diagram of the FLAT search algorithm and the LSH search algorithm.

[0037] Figure 5 A schematic diagram showing the bottleneck of CPU-based vector similarity search.

[0038] Figure 6A and Figure 6B An architectural diagram illustrating a CPU-based vector similarity search and a search according to an embodiment of the present disclosure.

[0039] Figure 7 A flowchart illustrating a search process on the host side according to an embodiment of the present disclosure is shown.

[0040] Figure 8 A schematic diagram illustrating index creation in a host according to an embodiment of the present disclosure.

[0041] Figure 9 A flowchart illustrating index creation in a host according to an embodiment of the present disclosure is shown.

[0042] Figure 10A schematic diagram illustrating performing a search in a host according to an embodiment of the present disclosure.

[0043] Figure 11 A flowchart illustrating a search process in a PNM device according to an embodiment of the present disclosure is shown.

[0044] Figure 12 A schematic diagram illustrating index creation or search execution in a PNM device according to an embodiment of the present disclosure.

[0045] Figure 13 The process of performing vector similarity calculation inside the PNM device according to an embodiment of the present disclosure is shown.

[0046] Figure 14 A schematic diagram illustrating a host according to an embodiment of the present disclosure.

[0047] Figure 15 A schematic diagram illustrating a PNM device according to an embodiment of the present disclosure is shown.

[0048] Figure 16 is a schematic diagram of a system to which a storage device is applied according to an embodiment of the present disclosure.

[0049] Figure 17 is a block diagram of a host storage system according to an embodiment of the present disclosure.

[0050] Figure 18 is a diagram of a data center to which a storage device is applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0052] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0053] It should be noted that the phrase "at least one of the items" in this disclosure includes three types of parallel situations: "any one of the items", "a combination of any multiple items of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. For another example, "performing at least one of step 1 and step 2" includes the following three parallel situations: (1) performing step 1; (2) performing step 2; and (3) performing steps 1 and 2.

[0054] The technical solution provided according to the embodiments of the present disclosure brings at least the following beneficial effects: search-related calculations are offloaded to the PNM device for execution, thereby significantly reducing data movement between the CPU and the memory, and freeing the CPU from a large number of search-related calculation tasks, enabling it to respond to other priority tasks more quickly; in addition, the use of storage devices combined with PNM devices improves the efficiency of data transmission, thereby improving the performance of search-related calculations of the vector database.

[0055] Figure 1 A conceptual diagram showing a vector database. Figure 1 , the vector database converts the original data set including unstructured data into a vector data set (hereinafter referred to as the data set) through an embedding model. Then, the data set is stored in the vector database. The similarity between vectors can be measured using mathematical metrics (for example, the Euclidean distance between vectors). There are many algorithms that support vector similarity search operations, such as brute force search (for example, FLAT), Local Sensitive Hashing (LSH), Inverted Index Quantization (IVF+ProductQuantization, IVF-PQ), Approximate Nearest Neighbors OhYeah (ANNOY), Hierarchical Navigable Small World (HNSW), etc. Based on vector similarity search, the vector database can identify the most similar data items in the vector data set. Vector databases can be used in recommendation systems (including recommending similar products in online shopping, finding similar pictures, or recommending similar videos on video websites), biological protein structure or gene sequence identification, financial fraud detection, Internet of Things data analysis, and other fields.

[0056] Figure 2 The following diagram shows the workflow of the vector database. Figure 2, the workflow of the vector database can be described by the following operations. Operation 1 includes preparing the original data set. Operation 2 includes using a pre-trained model to convert the original data set into a vector data set (called an embedding process) and store it in the vector database. Operation 3 includes converting the query object data into a vector using an embedding process (for example, the process of operation 2). Operation 4 includes searching the vector database for the data item that is most similar to the query vector. Regarding operation 4, for smaller vector data sets, a direct brute force search can be performed. As another example, for large vector data sets, the vector database can provide an indexing function, which can, for example, use a specified vector similarity search algorithm to pre-sort the vector data set, thereby narrowing the query range during actual query to speed up the query processing process. Operation 5 includes the vector database returning the top K (i.e., top K) results in the data set that are most similar to the query object.

[0057] Figure 3 Schematic diagram of the CPU-based vector similarity search process. Figure 3 , detailed explanation Figure 2 The vector similarity search process described in operation 4 of is performed. This process includes loading a vector dataset from a storage device (e.g., a solid-state drive (SSD)) into memory, and then having the CPU read the vectors in memory for similarity calculation. The CPU can perform similarity calculation based on various similarity search algorithms (e.g., FLAT, LSH, IVF-PQ, ANNOY, or HNSW) provided by a vector similarity search engine in the vector database.

[0058] Figure 4A and Figure 4B A schematic diagram comparing the FLAT search algorithm and the LSH search algorithm is shown. Figure 4A is a schematic diagram of the FLAT search algorithm, and Figure 4B This is a schematic diagram of the LSH search algorithm.

[0059] In FLAT, the similarity between each vector in the vector database and the query vector is calculated. When the vector database includes N entries (i.e., N vectors), Figure 4A The process shown in is repeated N times: the CPU reads the i-th vector (i.e., vector[i]) from the memory, calculates the similarity between the vector and the query vector, and writes the vector to the memory when the top K candidate vectors need to be updated. In LSH, as Figure 4B As shown in FIG, first, an index is created, and then the vectors in the vector database are mapped to different hash buckets through hash mapping. Then, a query is executed / searched only in the hash bucket corresponding to the query vector. In the indexing process, if the vector database includes N entries (i.e., N vectors), as shown in FIG. Figure 4BThe process shown is repeated N times. The CPU reads the i-th vector (i.e., vector[i]) from the memory, performs a hash calculation on the vector, and inserts the hash calculation result into the hash table (i.e., writes data to the memory). In the query process, if the hash bucket corresponding to the query vector includes M entries, as shown in Figure 4B The process shown is repeated M times: the CPU reads a vector (i.e., vector[j]) from memory, calculates the similarity between the vector and the query vector, and writes the vector to memory when the top K candidate vectors need to be updated.

[0060] It should be understood that the above search algorithms are merely examples, the present disclosure is not limited thereto, and other search algorithms may also follow the same pattern.

[0061] After analyzing the relevant technologies regarding the current CPU-based vector similarity search related calculations running in the CPU and memory, the problems in the relevant technologies include at least: the CPU frequently and repeatedly loads vectors from the memory and writes back the vector operation results, and a large number of vector similarity calculation tasks greatly increase the CPU workload.

[0062] Figure 5 A CPU-based vector similarity search bottleneck diagram is shown. Figure 5 As the size of vector datasets increases, it becomes a performance bottleneck for vector similarity searches. In vector similarity searches, large amounts of data must be moved between the CPU and memory while maintaining a relatively narrow data bandwidth. Furthermore, the CPU is burdened with a large number of vector similarity calculation tasks, impacting the overall system response time.

[0063] In order to fundamentally reduce the data exchange between the CPU and memory, and free the CPU from heavy vector similarity search-related computing tasks, the present invention utilizes the near memory processing characteristics of the Process Near Memory (PNM) device to offload the vector search-related computing tasks in the vector database from the CPU to the PNM device, and proposes a corresponding accelerated computing architecture.

[0064] PNM is a technology that integrates memory and logic chips into advanced integrated circuit packages. This reduces data movement between the CPU and memory by utilizing memory for computation. Compute Express Link (CXL)-PNM technology utilizes the CXL interface to establish a high-capacity, high-speed connection between the processor and memory. By performing computations closer to the memory, CXL-PNM reduces data transfer bottlenecks between the CPU and memory.

[0065] 6 to 6 Figure 18The search method, host, and PNM device according to the present disclosure are described in detail.

[0066] Figure 6A and Figure 6B A comparison between a CPU-based vector similarity search architecture and a search architecture according to an embodiment of the present disclosure is shown. Figure 6A It is the architecture of CPU-based vector similarity search in related technologies, and Figure 6B This is the architecture of the search according to the embodiment of the present disclosure. The vector similarity search based on the CPU is used as an example, but it should be understood that the architecture of the search of the present disclosure is not limited to similarity search. In the architecture of the search of the present disclosure, both vector similarity search based on the CPU ( Figure 6B The left route in the figure) can offload the search-related computational tasks to the PNM device (e.g., CXL-PNM device) ( Figure 6B The right route in the vector database). A PNM-oriented interface (IF) can be added to the vector similarity search engine of the vector database, and the host can call the PNM device through the PNM-oriented interface to perform search-related calculations. In the search architecture disclosed in the present invention, a CXL-PNM-SSD (Solid State Disk) device can be used. The CXL-PNM-SSD device includes a CXL-PNM composed of a device memory and a computing unit, and an SSD containing a flash memory (Flash), wherein the SSD stores the vector data set in the Flash. When the CXL-PNM performs search-related calculations, the vector data set stored in the Flash can be loaded into the device memory, and the computing unit performs similarity calculations on the loaded vector data set according to the similarity search algorithm. Since the SSD and CXL-PNM are integrated into a CXL-PNM-SSD device, the data transmission between the Flash and the CXL-PNM can be considered as internal transmission of the device, thereby improving the data transmission speed between the Flash and the device memory.

[0067] The present disclosure may consist of two modules, deployed on a host and a PNM device (e.g., a CXL-PNM device), respectively: a host module that provides a set of software interfaces within a vector database; and a PNM device module that implements vector search-related computations within the PNM device. The host module may be responsible for invoking the PNM device to perform vector search-related computations (e.g., similarity search-related computations), while the PNM device module may be responsible for interacting with the host interface, accessing device memory, and performing vector search-related computations.

[0068] In an embodiment, a CPU-based vector similarity search ( Figure 6B The left route in Figure 6BThe right side of the same storage device (Flash). However, Figure 6B The left path in the example can use a separate storage device, such as another SSD (such as Figure 8 and Figure 10 It should be understood that the above-mentioned search architecture according to the embodiment of the present disclosure is only an example, and the present disclosure is not limited thereto.

[0069] Figure 7 A flowchart of a host-side search method according to an embodiment of the present disclosure is shown.

[0070] Reference Figure 7 , in operation S710, a near memory processing (PNM) device is called to perform search-related calculations on a vector data set in a vector database.

[0071] In an embodiment of the present disclosure, the host side may call the PNM device to offload search-related computations on the vector dataset to the PNM device for execution. Here, the search-related computations may include at least one of establishing an index of the vector dataset and / or performing a search on the vector dataset.

[0072] According to an embodiment of the present disclosure, a call indication may be sent to a PNM device through an interface provided by a vector database (eg, a PNM-oriented interface), so as to call the PNM device to perform search-related calculations on a vector data set.

[0073] In an embodiment of the present disclosure, a PNM-oriented interface may be added to the vector similarity search engine of the vector database, and the host side may call the PNM device through the PNM-oriented interface to perform search-related calculations.

[0074] According to an embodiment of the present disclosure, when establishing an index, index parameters may be provided to a PNM device via an indexing interface, invoking the PNM device to index vectors in a vector dataset. The established index may be used to perform searches on the vector dataset. The index parameters may include an index type. When the index type is an index type specific to the PNM device, the PNM device may be invoked to index the vectors in the vector dataset.

[0075] In an embodiment of the present disclosure, a first host-side interface may be provided in the vector database as an interface for invoking a PNM device to establish an index for a vector dataset. The interface parameters (i.e., index parameters) may include an index type. The vector database may provide an index type when creating an index. In the present disclosure, when establishing an index, the vector database may provide a new PNM_* index type based on an existing index type to instruct the vector database to invoke a PNM device (e.g., a CXL-PNM device) instead of a CPU to complete the indexing operation.

[0076] Figure 8A schematic diagram showing the process of indexing on the host side according to an embodiment of the present disclosure is shown. The following PNM device is described using the CXL-PNM device as an example. Figure 8 In the embodiment, CXL-PNM-SSD device is used to improve the data transmission speed between Flash and device memory, but the present disclosure is not limited thereto. Figure 8 , the host side performs operations via the vector database interface. In the indexing operation, the host side calls the PNM device (e.g., CXL-PNM device) to index the vectors in the vector data set according to the index type. Specifically, a new PNM_* index type (e.g., PNM_FLAT or PNM_IVF_PQ) is provided on the basis of the existing index type to indicate that the PNM device is called when indexing. In an embodiment, when the index type is a PNM_* index type, the host side may call the PNM device to index the vectors in the vector data set. In the same or other embodiments, when the index type is an existing index type (i.e., not a PNM_* index type), the CPU on the host side may index the vectors in the vector data set.

[0077] When the index type is a PNM_* index type, the host uses the indexing interface (e.g., CreateIndexPNM) to offload the indexing task to the PNM device. When the index type is a PNM_* index type, the vector dataset in the vector database can be stored in a storage device integrated with the PNM device by default, such as an SSD or Flash. The PNM device loads the vector dataset stored in Flash into the device memory. The PNM device indexes the vectors in the vector dataset loaded into the device memory based on the index type provided through the indexing interface.

[0078] In the same or other embodiments, when the index type is not a PNM_* index type, the vector dataset in the vector database is stored in a separate storage device (e.g., another SSD) by default. The host CPU loads the vector dataset stored in the storage device into memory, and the CPU indexes the vectors in the vector dataset loaded into memory.

[0079] Figure 9 FIG. 1 shows a flow chart of indexing on the host side according to an embodiment of the present disclosure. Figure 9The indexing process includes operations 901 and 902. In operation 901, the host side uses the indexing processing method (indexing operation) provided by the vector database based on the index type. If the index type is a PNM_* index type, operation 2 is executed. In operation 2, the interface (e.g., CreateIndexPNM) for the PNM device to index the vector dataset is called to instruct the PNM device to complete the indexing and return the result to the host side. However, if the index type is not a PNM_* index type, the existing CPU-based processing is adopted.

[0080] In embodiments of the present disclosure, indexing can involve pre-processing the vectors in a vector dataset, such as sorting or classification, to narrow the search scope or reduce computational complexity during actual queries, thereby accelerating the query process. Furthermore, different similarity search methods may result in different methods for indexing vectors in a vector dataset. For example, consider the LSH search algorithm shown in Figure 4. The basic idea of ​​LSH is to hash the data items in a dataset into buckets, ensuring that similar data items are mapped to the same bucket with a high probability. During a query, the query item is hashed to obtain its bucket number, and then the query item is compared with the data items in the same bucket to quickly find data items similar to the query item. Through hash mapping operations, LSH transforms the problem of finding similar data items in a very large set into the problem of finding similar data items in a very small set, significantly reducing the computational effort. LSH can also use multiple hash tables to increase accuracy. Each hash table is composed of different hash functions and generates different hash buckets. During a query, all data items in the buckets of the different hash tables into which the query item falls are used as a candidate set, and then data items similar to the query item are searched within the candidate set. For LSH, the operation of indexing the vectors in the vector data set may include: mapping the vectors in the vector database to different hash buckets through hash mapping, and storing (inserting) the vectors into corresponding hash buckets in the hash table.

[0081] For another example, consider the IVF-PQ search algorithm, which combines the techniques of inverted index (Inverted File) and product quantization (Product Quantization). In this technology, for each vector, the inverted index stores a list of data including the vector, which makes it possible to quickly locate data containing similar features when querying, while product quantization reduces the complexity of storage and calculation and helps to speed up the search by decomposing high-dimensional vectors into smaller sub-vectors and independently quantizing each sub-vector. For the IVF-PQ search algorithm, the operation of indexing vectors in a vector data set may include: for each high-dimensional vector in the vector data set, using product quantization to map these high-dimensional vectors to a low-dimensional codebook; then constructing an inverted index on the low-dimensional codebook, and establishing an inverted list for the data corresponding to each codebook.

[0082] It should be understood that indexing is described here using the LSH search algorithm and the IVF-PQ search algorithm as examples, but the indexing operation is not limited thereto, and other search algorithms, such as FLAT ANNOY and HNSW, also have corresponding indexing operations, which will not be discussed in this disclosure.

[0083] According to an embodiment of the present disclosure, when performing a search on a vector dataset, search parameters may be provided to a PNM device via a search interface, invoking the PNM device to search for a query vector within the vector dataset. The search parameters may include the query vector and the vector dataset storage location. Furthermore, if the vector dataset storage location is in a storage device integrated with the PNM device, the PNM device may be invoked to search for the query vector within the vector dataset.

[0084] In an embodiment of the present disclosure, a second host-side interface may be provided in the vector database as an interface for calling a PNM device to perform a vector search. Generally, there are two scenarios for performing a vector search in a vector database, searching in an indexed vector dataset or searching directly on an original vector dataset (not indexed). When searching in an indexed vector dataset, since the vector dataset has been indexed and sorted, the search can more easily narrow the scope and quickly obtain results. Therefore, the present disclosure introduces the use of a PNM device to accelerate processing when searching directly on an original vector dataset. It should be understood that the method of the present disclosure can also be applied to an indexed vector dataset.

[0085] Figure 10 Schematic diagram of the host side performing search according to an embodiment of the present disclosure is shown. The following PNM device is described by taking the CXL-PNM device as an example, and in Figure 10In the embodiment, CXL-PNM-SSD device is also used to improve the data transmission speed between Flash and device memory, but the present disclosure is not limited thereto. Figure 10 , the host side performs a search operation via the vector database interface. During the search operation, the host side calls the PNM device to search for the query vector in the vector dataset according to the storage location of the vector dataset. The specific process on the host side may include an operation, wherein the search process detects the storage location of the vector dataset; and if the storage location of the vector dataset is in a storage device integrated with the PNM device, that is, an SSD (e.g., Flash), the search interface (e.g., SearchPNM) is used to instruct the CXL-PNM device to complete the search and return the results to the host side. The search parameters may include: the query vector and the vector dataset storage location. The CXL-PNM device may load the vector dataset stored in the Flash into the device memory according to the vector dataset storage location. The vector dataset storage location may include an address in the storage device integrated with the PNM device (i.e., an address in the Flash). The CXL-PNM device may then search for the query vector on the vector dataset loaded into the device memory. However, according to an embodiment, if the vector dataset storage location is in a separate storage device (for example, another SSD that is not integrated with the PNM device), the CPU on the host side loads the vector dataset stored in the storage device into the memory, and the CPU searches for the query vector on the vector dataset loaded into the memory, that is, using existing CPU-based processing.

[0086] return Figure 7 In operation S720, a result associated with the search-related calculation returned by the PNM device is received.

[0087] According to an embodiment of the present disclosure, when indexing a vector dataset, an indexing completion result returned by a PNM device is received; and when searching on the vector dataset, a search result related to a query vector is received by the PNM device.

[0088] In an embodiment of the present disclosure, when indexing a vector dataset, the PNM device may return an indexing completion result indicator, indicating whether the indexing is complete or incomplete. Furthermore, when performing a search on the vector dataset (e.g., using a similarity search algorithm), multiple vectors that are most similar to the query vector may be returned.

[0089] According to the host-side search method described above, search-related computations are offloaded to the PNM device, significantly reducing data movement between the CPU and memory and freeing the CPU from the heavy workload of search-related computations. This allows the CPU to respond more quickly to other priority tasks. Furthermore, the use of storage devices in conjunction with the PNM device improves data transmission efficiency and enhances the performance of search-related computations in the vector database.

[0090] Figure 11 A flow chart of a search method on a PNM device side according to an embodiment of the present disclosure is shown.

[0091] Reference Figure 11 , in operation S1110, in response to receiving a call instruction from the host, a search-related calculation is performed on the vector data set in the vector database.

[0092] In an embodiment of the present disclosure, upon receiving a call instruction from a host, the PNM device may replace the CPU to perform search-related calculations on a vector dataset in a vector database. The search-related calculations may include at least one of establishing an index for the vector dataset and performing a search on the vector dataset.

[0093] According to an embodiment of the present disclosure, in response to a call indication sent by a host via an interface provided by a vector database, a search-related calculation is performed on a vector data set. Based on index parameters provided by the host using an interface for indexing, vectors in the vector data set are indexed, and the established index is used to perform a search on the vector data set. The index parameters used for indexing include: an index type. In an embodiment, based on search parameters provided by the host using an interface for searching, a query vector is searched on the vector data set, and the search parameters include: a query vector and a storage location of the vector data set.

[0094] In an embodiment of the present disclosure, a host sends a call instruction via an indexing interface (e.g., CreateIndexPNM) or a search interface (e.g., SearchPNM) provided by a vector database and provides index parameters and search parameters to a PNM device, respectively. After receiving the corresponding call instruction, the PNM device indexes the vectors in the vector dataset based on the index parameters, or searches for a query vector on the vector dataset based on the search parameters.

[0095] According to an embodiment of the present disclosure, a vector dataset is loaded from a storage device integrated with a PNM device into a device memory of the PNM device, and vectors in the vector dataset loaded into the device memory of the PNM device are indexed according to a similarity search method indicated by an index type. After the indexing is completed, the index is written from the device memory of the PNM device to the storage device integrated with the PNM device.

[0096] According to an embodiment of the present disclosure, the search parameters may further include a similarity search method. In an embodiment, the PNM device may load the vector dataset from a storage device integrated with the PNM device into a device memory of the PNM device based on a storage location of the vector dataset, and search the vector dataset loaded into the device memory of the PNM device for multiple vectors with the highest similarity to the query vector according to the similarity search method.

[0097] Figure 12 FIG. 1 is a schematic diagram showing indexing or search execution on the PNM device side according to an embodiment of the present disclosure. Figure 12 In the embodiment, the CXL-PNM-SSD device is used for illustration, but the present disclosure is not limited thereto. Figure 12 The PNM device-side workflow may include: a first operation, in which the PNM device receives a processing request (e.g., indexing / performing a search); a second operation, in which the PNM device loads a vector dataset from a storage device (e.g., a storage device integrated with the PNM device, i.e., an SSD, specifically Flash here) into a device memory; and a third operation, in which the PNM device computing unit, when the processing request is indexing, indexes the vectors in the loaded vector dataset, and when performing a search, searches for a query vector on the loaded vector dataset.

[0098] The computational unit within the PNM device can portably implement widely used vector similarity search algorithms (such as FLAT, IVF-PQ, and HNSW), and the computational unit can optimize acceleration for specific search algorithms. Compared to general-purpose CPUs, this optimized and accelerated computational unit can improve the efficiency of search-related calculations. When indexing, the computational unit of the PNM device can access and read vectors from device memory and index the vectors in the vector dataset based on the similarity search method specified by the index type included in the index parameter. For example, if the index type is PNM_IVF_PQ, the similarity search method is IVF_PQ. The computational unit of the PNM device can index the vectors in the vector dataset based on IVF_PQ. After indexing is complete, the computational unit of the PNM device can write the created index from the device memory of the PNM device to a storage device integrated with the PNM device. After the index is created, the created index can be used for subsequent searches on the vector dataset. Specifically, during subsequent searches (queries), searches can be performed based on the created index, and this search task can be performed by the CPU or offloaded to the PNM device. As an example, after the index is established according to LSH, that is, the vectors in the vector data set have been hash-mapped to different hash buckets in the hash table, and the subsequent search (query) for the query vector can be performed in a certain hash bucket that has been mapped. In addition to the index type, the index parameters may also include other parameters, such as the storage location (of the vector data set) indicating the storage address of the vector data set in Flash, etc., but the present disclosure is not limited to this. In addition, the similarity search method specified by the index type may include FLAT, LSH, IVF-PQ, ANNOY and HNSW, etc., but the present disclosure is not limited to this.

[0099] When performing a search, the computing unit of the PNM device may access and read the vectors in the device memory and calculate the similarity according to the similarity search method included in the search parameters. The computing unit of the PNM device may then write back the calculation results to the device memory. For example, when the similarity search method is FLAT, the computing unit may search for multiple vectors with the highest similarity to the query vector in the vector dataset according to FLAT, and write back the multiple vectors with the highest similarity to the device memory. In addition, in addition to the query vector, the vector dataset storage location, and the similarity search method, the search parameters may also include other parameters, such as K (where K is a positive integer) indicating the return of the top K most similar vectors, but the present disclosure is not limited thereto. In addition, the similarity search methods included in the search parameters may include FLAT, LSH, IVF-PQ, ANNOY, and HNSW, but the present disclosure is not limited thereto. Moreover, when the similarity search method as the search parameter is missing, a default similarity search method may be provided, such as FLAT.

[0100] Figure 13 The detailed process of executing vector similarity calculation inside the PNM device according to the embodiment of the present disclosure is shown. Figure 13 , Figure 3 The process in the figure describes a case where the host initiates a vector similarity search and the CXL-PNM device executes the FLAT vector similarity search algorithm on the vector data set. However, it should be understood that the present disclosure is not limited thereto.

[0101] Operation starts: The host initiates a vector similarity search, and the CXL-PNM device executes the FLAT vector similarity search algorithm on the vector dataset.

[0102] Operation 1301: The computing unit of the PNM device reads a vector from the vector data set as the current vector. However, if all vectors in the vector data set have been processed, the process ends.

[0103] Operation 1302: The calculation unit of the PNM device calculates the similarity between the current vector and the query vector.

[0104] Operation 1303: The computing unit of the PNM device reads one of the similarity results of the current candidate (ie, the top-k most similar vectors).

[0105] Operation 1304: The calculation unit of the PNM device compares the read similarity result of the current candidate with the similarity result of the current vector.

[0106] Operation 1305: If the current vector is more similar than the current candidate, the calculation unit of the PNM device updates the current vector to the candidate result. If not, jump to operation 1303.

[0107] It should be understood that the present disclosure uses FLAT as an example for illustration, but the search method and search architecture of the present disclosure can be applied to other search algorithms, and the present disclosure is not limited thereto.

[0108] return Figure 11 , in operation S1120, after the search-related calculation is completed, the results associated with the search-related calculation are returned to the host.

[0109] According to an embodiment of the present disclosure, when an index is created for a vector dataset, a result indicating that the index is created can be returned to the host. Alternatively, when a search is performed on the vector dataset, search results related to the query vector can be returned to the host.

[0110] In an embodiment of the present disclosure, when indexing a vector dataset, the PNM device may return an indexing completion result indicator to indicate whether the indexing is complete or incomplete. Furthermore, when performing a search on a vector dataset, for example, using a similarity search method, multiple vectors most similar to the query vector may be returned.

[0111] According to the search method described above on the PNM device side, search-related calculations are offloaded to the PNM device for execution, significantly reducing data movement between the CPU and memory and freeing the CPU from a large number of search-related calculation tasks. This enables the CPU to respond more quickly to other priority tasks. In addition, the use of storage devices combined with PNM devices improves data transmission efficiency and enhances the performance of search-related calculations in the vector database.

[0112] Figure 14 A schematic diagram illustrating a host according to an embodiment of the present disclosure.

[0113] Reference Figure 14 , the host 1400 may include a memory 1410 (e.g., a memory) and a processor 1420 (e.g., a CPU), wherein the memory 1410 may store instructions; and when the instructions are executed by the processor 1420, the processor 1420 is prompted to perform the following operations: call a near memory processing PNM device to perform search-related calculations on a vector data set in a vector database; and receive results of the search-related calculations returned by the PNM device.

[0114] According to an embodiment of the present disclosure, when the instruction is executed by the processor 1420, the processor 1420 may be prompted to perform the following operations: sending a call instruction to the PNM device through the interface provided by the vector database, and calling the PNM device to perform search-related calculations on the vector data set.

[0115] According to an embodiment of the present disclosure, when the instruction is executed by the processor 1420, the processor 1420 may be prompted to perform the following operations: in the case of establishing an index for a vector dataset, index parameters are provided to the PNM device through the index establishment interface, and the PNM device is called to index the vectors in the vector dataset, wherein the established index is used to perform a search on the vector dataset, and the index parameters include: index type; or in the case of performing a search on the vector dataset, search parameters are provided to the PNM device through the search interface, and the PNM device is called to search for a query vector on the vector dataset, wherein the search parameters include: query vector and vector dataset storage location.

[0116] According to an embodiment of the present disclosure, when the instruction is executed by the processor 1420, it may prompt the processor 1420 to perform the following operations: when the index type is an index type about the PNM device, call the PNM device to index the vectors in the vector dataset; and when the vector dataset storage location is in a storage device integrated with the PNM device, call the PNM device to search for the query vector on the vector dataset.

[0117] According to an embodiment of the present disclosure, when the instruction is executed by the processor 1420, it may prompt the processor 1420 to perform the following operations: in the case of establishing an index for a vector data set, receive the indexing completion result returned by the PNM device; or in the case of performing a search on the vector data set, receive the search results regarding the query vector returned by the PNM device.

[0118] Figure 15 A schematic diagram illustrating a PNM device according to an embodiment of the present disclosure is shown.

[0119] Reference Figure 15 The PNM device 1500 includes a device memory 1510 and a computing unit 1520, wherein the computing unit 1520 can perform search-related calculations on a vector data set in a vector database in response to receiving a call indication from a host; and after the search-related calculations are completed, return the results of the search-related calculations to the host.

[0120] According to an embodiment of the present disclosure, the computing unit 1520 may include one or more processors and may perform search-related computations on a vector dataset in response to a call instruction sent by a host via an interface provided by the vector database.

[0121] According to an embodiment of the present disclosure, the computing unit 1520 may index vectors in a vector dataset based on index parameters provided by a host using an interface for indexing, wherein the established index is used to perform a search on the vector dataset, and the index parameters include: an index type; or search for a query vector on the vector dataset based on search parameters provided by a host using an interface for searching, wherein the search parameters include: a query vector and a storage location of the vector dataset.

[0122] According to an embodiment of the present disclosure, the computing unit 1520 may load a vector dataset from a storage device integrated with the PNM device into the device memory 1510, index the vectors in the vector dataset loaded into the device memory 1510 according to the index type, and after the indexing is completed, write the established index from the device memory 1510 to the storage device integrated with the PNM device.

[0123] According to an embodiment of the present disclosure, the search parameters may also include: a similarity search method, and the calculation unit 1520 may load the vector dataset from a storage device integrated with the PNM device into the device memory 1510 according to the storage location of the vector dataset, and search for multiple vectors with the highest similarity to the query vector on the vector dataset loaded into the device memory 1510 according to the similarity search method.

[0124] According to an embodiment of the present disclosure, the computing unit 1520 may return the indexing completion result to the host when indexing the vector dataset; or return the search results about the query vector to the host when searching on the vector dataset.

[0125] As described above, in the host and PNM device of the present invention, search-related calculations are offloaded to the PNM device for execution, thereby significantly reducing data movement between the CPU and memory, and freeing the CPU from a large number of search-related calculation tasks, enabling it to respond to other priority tasks more quickly; in addition, the use of storage devices combined with PNM devices improves the efficiency of data transmission, thereby improving the performance of search-related calculations of the vector database.

[0126] Figure 16 A schematic diagram of a system 1000 to which a storage device is applied according to an embodiment.

[0127] Figure 16 The system 1000 may be, for example, a mobile system such as a portable communication terminal (e.g., a mobile phone), a smartphone, a tablet personal computer (PC), a wearable device, a healthcare device, or an Internet of Things (IoT) device. However, Figure 16 The system 1000 is not limited thereto, and may be, for example, a PC, a laptop computer, a server, a media player, or an automotive device (eg, a navigation device).

[0128] Reference Figure 16 The system 1000 may include a main processor 1100, memories (e.g., 1200a and 1200b), and storage devices (e.g., 1300a and 1300b). In addition, the system 1000 may include at least one of an image capture device 1410, a user input device 1420, a sensor 1430, a communication device 1440, a display 1450, a speaker 1460, a power supply 1470, and a connection interface 1480.

[0129] The main processor 1100 may control all operations of the system 1000, including, for example, operations of other components included in the system 1000. The main processor 1100 may be implemented as, for example, a general-purpose processor, a dedicated processor, an application processor, or the like.

[0130] The main processor 1100 may include at least one central processing unit (CPU) core 1110 and a controller 1120 for controlling memories 1200a and 1200b and / or storage devices 1300a and 1300b. In some exemplary embodiments, the main processor 1100 may further include an accelerator 1130, which is a dedicated circuit for high-speed data operations such as, for example, artificial intelligence (AI) data operations. The accelerator 1130 may include, for example, a graphics processing unit (GPU), a neural processing unit (NPU), and / or a data processing unit (DPU), and may be implemented as a chip physically separated from other components of the main processor 1100.

[0131] Memories 1200a and 1200b may serve as main storage devices of system 1000. Each of memories 1200a and 1200b may further include volatile memory, such as static random access memory (SRAM) and / or dynamic random access memory (DRAM), etc., but according to embodiments, memories 1200a and 1200b may each include non-volatile memory, such as flash memory, phase change random access memory (PRAM), and / or resistive random access memory (RRAM), etc. Memories 1200a and 1200b may be implemented in the same package as main processor 1100. Memory 1200b may also be a PNM device. For example, memory 1200a may be implemented in the same package as main processor 1100, while memory 1200b may be a PNM device external to the main processor, and the PNM device may also be integrated with a storage device (e.g., an SSD).

[0132] The memory devices 1300a and 1300b may function as non-volatile memory devices configured to store data regardless of whether power is supplied, and may have a larger storage capacity than the memories 1200a and 1200b. The memory devices 1300a and 1300b may include memory controllers (STRG CTRL) 1310a and 1310b, respectively, and non-volatile memories (NVMs) 1320a and 1320b, respectively, configured to store data under the control of the memory controllers 1310a and 1310b. Although the NVMs 1320a and 1320b may include V-NAND flash memories having a two-dimensional (2D) or three-dimensional (3D) structure, the NVMs 1320a and 1320b may include other types of NVMs, such as, for example, PRAM and / or RRAM.

[0133] The storage devices 1300a and 1300b may be physically separated from the main processor 1100 and included in the system 1000, or may be implemented in the same package as the main processor 1100. The storage devices 1300a and 1300b may be solid-state devices (SSDs) or memory cards and may be removably coupled to other components of the system 100 via an interface such as the connection interface 1480 described further below. The storage devices 1300a and 1300b may be devices to which a standard protocol such as Universal Flash Storage (UFS), embedded MultiMediaCard (eMMC), or NVMe is applied. However, the storage devices 1300a and 1300b are not limited thereto.

[0134] The image capturing device 1410 can capture still images or moving images and may include, for example, a camera, a camcorder, and / or a webcam.

[0135] The user input device 1420 may receive various types of data input by a user of the system 1000 and may include, for example, a touch pad, a keypad, a keyboard, a mouse, a microphone, and the like.

[0136] The sensor 1430 can detect various types of physical quantities that can be obtained from outside the system 1000 and convert the detected physical quantities into electrical signals. The sensor 1430 may include, for example, a temperature sensor, a pressure sensor, an illumination sensor, a position sensor, an acceleration sensor, a biosensor, and / or a gyroscope sensor.

[0137] The communication device 1440 may transmit and receive signals between other devices outside the system 1000 according to various communication protocols. The communication device 1440 may include, for example, an antenna, a transceiver, or a modem.

[0138] Display 1450 and speaker 1460 may serve as output devices configured to output visual information and auditory information, respectively, to a user of system 1000 .

[0139] The power supply device 1470 may appropriately convert power supplied from a battery embedded in the system 1000 and / or an external power source, and supply the converted power to each component of the system 1000 .

[0140] The connection interface 1480 may provide a connection between the system 1000 and an external device that is connected to the system 1000 and is capable of transmitting and receiving data to and from the system 1000. The connection interface 1480 may be implemented by using various interface schemes, such as, for example, Advanced Technology Attachment (ATA), Serial ATA (SATA), External Serial ATA (e-SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), Peripheral Component Interconnect (PCI), PCI express (PCIe), NVMe, IEEE 1394, Universal Serial Bus (USB) interface, Secure Digital (SD) card interface, MultiMediaCard (MMC) interface, Embedded MultiMediaCard (eMMC) interface, UFS interface, Embedded UFS (eUFS) interface, and Compact Flash (CF) card interface, etc.

[0141] According to an embodiment of the present disclosure, a system (e.g., 1000) applying a near memory processing (PNM) device is provided, including: a main processor (e.g., 1100); a memory (e.g., 1200a); a PNM device (e.g., 1200b); and a storage device (e.g., 1300a and 1300b), wherein the PNM device can execute the search method applied to the PNM device as described above.

[0142] Figure 17 is a block diagram of a host storage system 10 according to an embodiment.

[0143] The host storage system 10 may include a host 100 and a storage device 200. The storage device 200 may include a memory controller 210 and an NVM 220. According to an embodiment, the host 100 may include a host controller 110 and a host memory 120. The host memory 120 may be used as a buffer memory configured to temporarily store data to be transmitted to or received from the storage device 200.

[0144] The storage device 200 may include a storage medium configured to store data in response to a request from the host 100. As an example, the storage device 200 may include at least one of an SSD, an embedded memory, and a removable external memory. When the storage device 200 is an SSD, the storage device 200 may be a device that complies with the NVMe standard. When the storage device 200 is an embedded memory or an external memory, the storage device 200 may be a device that complies with the UFS standard or the eMMC standard. Both the host 100 and the storage device 200 may generate a packet according to the adopted standard protocol and may send the packet.

[0145] When the NVM 220 of the storage device 200 includes flash memory, the flash memory may include a 2D NAND memory array or a 3D (or vertical) NAND (VNAND) memory array. As another example, the storage device 200 may include various other types of NVM. For example, the storage device 200 may include magnetic random access memory (MRAM), spin transfer torque MRAM, conductive bridge RAM (CBRAM), ferroelectric RAM (FRAM), PRAM, RRAM, and various other types of memory.

[0146] According to an exemplary embodiment, the host controller 110 and the host memory 120 may be implemented as separate semiconductor chips. Alternatively, in some exemplary embodiments, the host controller 110 and the host memory 120 may be integrated into the same semiconductor chip. As an example, the host controller 110 may be any one of a plurality of devices included in an application processor (AP). The AP may be implemented as, for example, a system on a chip (SoC). In addition, the host memory 120 may be an embedded memory included in the AP or a memory device external to the AP. The host may also include a host memory 120 implemented external to the host, for example, the host memory 120 may be a near memory processing (PNM) device external to the host, and the PNM device may also be integrated with a storage device (e.g., an SSD).

[0147] The host controller 110 may manage an operation of storing data (eg, write data) of the buffer area of ​​the host memory 120 in the NVM 220 or an operation of storing data (eg, read data) of the NVM 220 in the buffer area.

[0148] The memory controller 210 may include a host interface 211, a memory interface 212, a CPU 213, a flash translation layer (FTL), a packet manager 215, a buffer memory 216, an error correction code (ECC) engine 217, and an advanced encryption standard (AES) engine 218. The memory controller 210 may further include a working memory in which the FTL 214 is loaded. The CPU 213 may execute the FTL 214 to control data write and read operations on the NVM 220.

[0149] The host interface 211 can transmit and receive packets to and from the host 100. The packets transmitted from the host 100 to the host interface 211 may include commands or data to be written to the NVM 220, etc. The packets transmitted from the host interface 211 to the host 100 may include responses to commands or data read from the NVM 220, etc. The memory interface 212 can transmit data to be written to the NVM 220 to the NVM 220 or receive data read from the NVM 220. The memory interface 212 may be configured to comply with a standard protocol such as, for example, Toggle or Open NAND Flash Interface (ONFI).

[0150] The FTL 214 can perform various functions, such as, for example, address mapping, wear leveling, and garbage collection. The address mapping operation may be an operation that converts a logical address received from the host 100 into a physical address for actually storing data in the NVM 220. The wear leveling operation may be a technique for preventing or reducing excessive degradation of a particular block by ensuring uniform use of the blocks of the NVM 220. For example, the wear leveling operation may be implemented using firmware techniques that balance the erase counts of physical blocks. The garbage collection operation may be a technique for ensuring available capacity in the NVM 220 by erasing existing blocks after copying valid data from the existing blocks to new blocks.

[0151] The packet manager 215 may generate a packet according to a protocol agreed with an interface of the host 100, or parse various types of information from a packet received from the host 100. In addition, the buffer memory 216 may temporarily store data to be written to or read from the NVM 220. Although the buffer memory 216 may be a component included in the memory controller 210, in an embodiment, the buffer memory 216 may be provided outside the memory controller 210.

[0152] The ECC engine 217 may perform error detection and correction operations on read data read from the NVM 220. For example, the ECC engine 217 may generate parity bits for write data to be written to the NVM 220, and the generated parity bits may be stored together with the write data in the NVM 220. During reading of data from the NVM 220, the ECC engine 217 may correct errors in the read data by using the read data and the parity bits read from the NVM 220, and output the error-corrected read data.

[0153] The AES engine 218 may perform at least one of an encryption operation and a decryption operation on data input to the memory controller 210 by using a symmetric key algorithm.

[0154] According to an embodiment of the present disclosure, a host storage system (e.g., 10) is provided, including: a host (e.g., 100); and a near memory processing (PNM) device, wherein the host can execute the search method applied to the host as described above, and the PNM device can execute the search method applied to the PNM device as described above.

[0155] Figure 18 is a diagram of a data center 3000 to which a storage device is applied, according to an exemplary embodiment.

[0156] Platform - Server (Application / Storage)

[0157] Reference Figure 18 , the data center 3000 can be a facility that collects various types of data and provides services, and can be called a data storage center. The data center 3000 can be a system for operating a search engine and a database, and can be a computing system used by a company (such as a bank) or a government agency. The data center 3000 may include application servers 3100 to 3100n and storage servers 3200 to 3200m, where n and m are positive integers. According to an exemplary embodiment, the number of applications 3100 to 3100n and the number of storage servers 3200 to 3200m can be selected differently. The number of application servers 3100 to 3100n and the number of storage servers 3200 to 3200m can be different from each other.

[0158] The application server 3100 or the storage server 3200 may include processors 3110 and 3210, at least one of memories 3120 and 3220, at least one of switches 3130 to 3130n, at least one of network interface cards (NICs) 3140 to 3140n and 3240 to 3240m, at least one of DRAMs 3253 to 3253m, and at least one of controllers 3251 to 3251m. The storage server 3200 will now be described as an example. The processor 3210 may control all operations of the storage server 3200, access the memory 3220, and execute instructions and / or data loaded into the memory 3220. The memory 3220 may be, for example, double data rate synchronous DRAM (DDR SDRAM), high bandwidth memory (HBM), a hybrid memory cube (HMC), a dual in-line memory module (DIMM), an Optane DIMM, or a non-volatile DIMM (NVM DIMM). In some exemplary embodiments, the number of processors 3210 and memories 3220 included in the storage server 3200 may be selected differently. In one exemplary embodiment, the processors 3210 and memories 3220 may provide a processor-memory pair. In one exemplary embodiment, the number of processors 3210 and the number of memories 3220 may differ. The processor 3210 may include a single-core processor or a multi-core processor. The above description of the storage server 3200 may similarly apply to the application server 3100. In some embodiments, the application server 3100 may not include a storage device 3150. The storage server 3200 may include at least one storage device 3250. Depending on the embodiment, the number of storage devices 3250 included in the storage server 3200 may be selected differently.

[0159] Platform part - Network

[0160] The application servers 3100 to 3100n can communicate with the storage servers 3200 to 3200m via a network 3300. The network 3300 can be implemented using Fibre Channel (FC) or Ethernet. In this case, FC can be a medium for relatively high-speed data transmission, and optical switches with high performance and high availability can be used. Depending on the access method of the network 3300, the storage servers 3200 to 3200m can be configured as file storage, block storage, or object storage.

[0161] In one embodiment, network 3300 may be a network dedicated to storage, such as a storage area network (SAN). For example, the SAN may be an FC-SAN, which uses an FC network and is implemented according to the FC protocol (FCP). As another example, the SAN may be an Internet Protocol (IP)-SAN, which uses a Transmission Control Protocol (TCP) / IP network and is implemented according to the SCSI over TCP / IP or Internet SCSI (iSCSI) protocol. In one embodiment, network 3300 may be a general-purpose network, such as a TCP / IP network. For example, network 3300 may be implemented according to protocols such as FC over Ethernet (FCoE), Network Attached Storage (NAS), and NVMe over Fabric (NVMe-oF).

[0162] Hereinafter, description will be mainly made of the application server 3100 and the storage server 3200. The description of the application server 3100 may be applied to another application server 3100n, and the description of the storage server 3200 may be applied to another storage server 3200m.

[0163] The application server 3100 can store data requested to be stored by a user or client in one of the storage servers 3200 to 3200m via the network 3300. In addition, the application server 3100 can obtain data requested to be read by a user or client from one of the storage servers 3200 to 3200m via the network 3300. For example, the application server 3100 can be implemented as a network server or a database management system (DBMS).

[0164] Application server 3100 can access memory 3120n or storage device 3150n included in another application server 3100n via network 3300. Alternatively, application server 3100 can access memory 3220 to 3220m or storage devices 3250 to 3250m included in storage servers 3200 to 3200m via network 3300. Thus, application server 3100 can perform various operations on data stored in application servers 3100 to 3100n and / or storage servers 3200 to 3200m. For example, application server 3100 can execute instructions for moving or copying data between application servers 3100 to 3100n and / or storage servers 3200 to 3200m. In this case, data can be moved from the storage devices 3250 to 3250m of the storage servers 3200 to 3200m through the memories 3220 to 3220m of the storage servers 3200 to 3200m or directly to the memories 3120 to 3120n of the application servers 3100 to 3100n. The data moved over the network 3300 may be encrypted for security or privacy. The storage server may include a PNM device, and the PNM device may also be integrated with a storage device (e.g., an SSD).

[0165] Organic Relationship-Interface Structure / Type

[0166] The storage server 3200 will now be described as an example. The interface 3254 can provide a physical connection between the processor 3210 and the controller 3251 and a physical connection between the network interface card (NIC) 3240 and the controller 3251. For example, the interface 3254 can be implemented using a direct attached storage (DAS) solution, wherein the storage device 3250 is directly connected to a dedicated cable. For example, the interface 3254 can be implemented using various interface solutions, such as ATA, SATA, e-SATA, SCSI, SAS, PCI, PCIe, NVMe, IEEE 1394, USB interface, SD card interface, MMC interface, eMMC interface, UFS interface, eUFS interface, and CF card interface.

[0167] The storage server 3200 may further include a switch 3230 and a network interconnect (NIC) 3240. The switch 3230 may selectively connect the processor 3210 to the storage device 3250 or selectively connect the NIC 3240 to the storage device 3250 under the control of the processor 3210.

[0168] In one embodiment, the NIC 3240 may include a network interface card and a network adapter. The NIC 3240 may be connected to the network 3300 via, for example, a wired interface, a wireless interface, a Bluetooth interface, or an optical interface. The NIC 3240 may include internal memory, a digital signal processor (DSP), and a host bus interface, and may be connected to the processor 3210 and / or the switch 3230 via the host bus interface. The host bus interface may be implemented as one of the above-described examples of the interface 3254. In one embodiment, the NIC 3240 may be integrated with at least one of the processor 3210, the switch 3230, and the storage device 3250.

[0169] Organic Relationship-Interface Operation

[0170] In the storage servers 3200 to 3200m or the application servers 3100 to 3100n, the processor may send a command to the storage devices 3150 to 3150n and 3250 to 3250m or the memories 3120 to 3120n and 3220 to 3220m and program or read data. In this case, the data may be data on which errors have been corrected by an ECC engine. The data may be data on which a data bus inversion (DBI) operation or a data masking (DM) operation has been performed and may include cyclic redundancy code (CRC) information. The data may be data encrypted for security or privacy.

[0171] Memory devices 3150 to 3150n and 3250 to 3250m can send control signals and command / address signals to NAND flash memory devices 3252 to 3252m in response to a read command received from a processor. Therefore, when reading data from NAND flash memory devices 3252 to 3252m, a read enable (RE) signal can be input as a data output control signal, thereby outputting the data to the DQ bus. The RE signal can be used to generate a data strobe signal DQS. Depending on the rising or falling edge of the write enable (WE) signal, command and address signals can be latched in the page buffer.

[0172] Product Section - Basic Operations of SSD

[0173] The controller 3251 can control all operations of the storage device 3250. In one embodiment, the controller 3251 may include an SRAM. The controller 3251 can write data to the NAND flash memory device 3252 in response to a write command, or read data from the NAND flash memory device 3252 in response to a read command. For example, the write command and / or read command can be provided by the processor 3210 of the storage server 3200, the processor 3210m of another storage server 3200m, or the processors 3110 and 3110n of the application servers 3100 and 3100n. The DRAM 3253 can temporarily store (or buffer) data to be written to or read from the NAND flash memory device 3252. The DRAM 3253 can also store metadata. Here, the metadata can be user data or data generated by the controller 3251 for managing the NAND flash memory device 3252. The storage device 3250 may include a secure element (SE) for security or privacy.

[0174] According to an exemplary embodiment of the present disclosure, a data center system (e.g., 3000) is provided, including: multiple application servers (3100 to 3100n); and multiple storage servers (e.g., 3200 to 3200m), wherein each storage server includes a near memory processing (PNM) device, wherein the PNM device can execute the search method applied to the PNM device as described above.

[0175] As conventional in the field of the present disclosure, embodiments are described and shown in the accompanying drawings according to functional blocks, units and / or modules. It will be appreciated by those skilled in the art that these blocks, units and / or modules are physically implemented by electronic (or optical) circuits (such as logic circuits, discrete components, microprocessors, hard-wired circuits, memory elements, wiring connections, etc.), which can be formed using semiconductor-based manufacturing techniques or other manufacturing techniques. In the case where blocks, units and / or modules are implemented by microprocessors or the like, software (e.g., microcode) can be used to program them to perform the various functions discussed herein, and they can optionally be driven by firmware and / or software. Alternatively, each block, unit and / or module can be implemented by dedicated hardware, or a combination of dedicated hardware that performs some functions and a processor that performs other functions (e.g., one or more programmed microprocessors and associated circuits).

[0176] According to an embodiment of the present disclosure, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the search method as described above is implemented.

[0177] According to an embodiment of the present disclosure, an electronic device is provided, including: a processor and a memory, wherein a computer program is stored. When the computer program is executed by the processor, the search method described above is implemented.

[0178] According to an embodiment of the present disclosure, a computer-readable storage medium storing a computer program may also be provided. When the program is run, the search method according to the embodiment of the present disclosure may be executed. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, for example, a multimedia card, a secure digital (SD) card and / or an ultra-fast digital (XD) card), a magnetic tape, a floppy disk, a magneto-optical data storage device, an optical data storage device, a hard disk, a solid state disk and / or any other device, any other device configured to store a computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and / or data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In one example, the computer program and any associated data, data files and / or data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0179] According to the search method, host and PNM device of the exemplary embodiments of the present disclosure, search-related calculations are offloaded to the PNM device for execution, thereby significantly reducing data movement between the CPU and the memory, and freeing the CPU from a large number of search-related calculation tasks, enabling it to respond to other priority tasks more quickly; in addition, the use of a storage device in combination with the PNM device improves the efficiency of data transmission, thereby improving the performance of search-related calculations of the vector database.

[0180] While the present disclosure has been particularly shown and described with reference to embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A search method, executed by at least one processor of a host, comprising: Calling a near memory processing (PNM) device to perform search-related calculations on a vector data set in a vector database; as well as Results associated with the search-related computations are received from the PNM device.

2. The method according to claim 1, wherein Calling the PNM device is based on sending a call indication to the PNM device through an interface provided by the vector database.

3. The method according to claim 2, wherein: The calling PNM device includes one of the following: Providing index parameters to the PNM device through an index establishment interface of the vector database, calling the PNM device to establish one or more indexes for the vectors in the vector data set, wherein the one or more indexes are used to perform a search on the vector data set, and the index parameters include: an index type; or The search parameters are provided to the PNM device through the search interface of the vector database, and the PNM device is called to search the query vector in the vector data set, wherein the search parameters include: the query vector and the storage location of the vector data set.

4. The method according to claim 3, wherein: The calling of the PNM device to establish one or more indexes for the vectors in the vector data set includes: Invoking the PNM device to create the one or more indexes for the vectors in the vector dataset based on the index type having a value associated with the PNM device, and The calling of the PNM device to search for the query vector in the vector data set includes: Based on the storage location of the vector dataset in a storage device integrated with the PNM device, the PNM device is called to search the query vector in the vector dataset.

5. The method according to claim 3, wherein: Receiving results associated with search-related calculations from the PNM device, including: Based on the PNM device being called to establish the one or more indexes for the vectors in the vector data set, receiving a result of completion of establishing the one or more indexes from the PNM device; or Based on the PNM device being invoked to search the vector dataset for the query vector, search results associated with the query vector are received from the PNM device.

6. A search method, performed by at least one processor of a near memory processing (PNM) device, the method comprising: In response to receiving a call indication from the host, performing a search-related calculation on a vector data set in a vector database; as well as After the search-related calculations are completed, results associated with the search-related calculations are returned to the host.

7. The method according to claim 6, wherein: The search related calculation is performed in response to a call instruction sent by the host via an interface provided by the vector database.

8. The method of claim 7, wherein: Performing a search-related calculation includes one of the following: establishing one or more indexes for vectors in the vector dataset based on index parameters provided by a host using an index establishment interface of a vector database, wherein the one or more indexes are used to perform a search on the vector dataset, and the index parameters include: an index type; or The vector data set is searched for a query vector based on search parameters provided by a host using a search interface of a vector database, wherein the search parameters include: a query vector and a vector data set storage location.

9. The method of claim 8, wherein: The step of establishing one or more indexes for the vectors in the vector dataset includes: loading the vector dataset from a storage device integrated with the PNM device into a device memory of the PNM device, establishing the one or more indexes for the vectors in the vector data set loaded into the device memory of the PNM device based on the similarity search method indicated by the index type, and The one or more indexes are written from the device memory of the PNM device to a storage device integrated with the PNM device.

10. The method of claim 8, wherein: The search parameters also include: similarity search method, and The step of searching the vector dataset for a query vector based on the search parameter includes: Based on the vector dataset storage location, loading the vector dataset from a storage device integrated with the PNM device into a device memory of the PNM device, and Based on a similarity search method, a plurality of vectors having similarity with the query vector are searched in the vector dataset loaded into the device memory of the PNM device.

11. The method of claim 8, wherein: Returns results associated with search-related computations to the host, including: Based on the PNM device being called to create the one or more indexes for the vectors in the vector data set, returning a result of completing the creation of the one or more indexes to the host; or Based on the PNM device being invoked to search for a query vector in the vector dataset, search results associated with the query vector are returned to the host.

12. A host comprising: Memory, which stores instructions, and At least one processor is configured to execute the instructions to perform the following operations: Invoking a near memory processing (PNM) device to perform search-related calculations on a vector data set in a vector database; and Receive the result associated with the search-related calculation returned by the PNM device.

13. A near memory processing (PNM) device, comprising: Device memory; as well as A computing unit, comprising at least one processor, wherein the at least one processor is configured to: In response to receiving a call indication from the host, performing a search-related calculation on a vector data set in a vector database; and After the search-related calculations are completed, results associated with the search-related calculations are returned to the host.

14. A system using a near memory processing (PNM) device, comprising: Main processor; Memory; PNM equipment; as well as storage device, The PNM device is configured to execute the search method according to any one of claims 6 to 11.

15. A host storage system comprising: Host; as well as Near Memory Processing (PNM) devices, The host is configured to: call a near memory processing (PNM) device to perform search-related calculations on a vector data set in a vector database; receive a result associated with the search-related calculations returned by the PNM device; and The PNM device is configured to: perform search-related calculations on a vector data set in a vector database in response to receiving a call instruction from a host; and return results associated with the search-related calculations to the host after the search-related calculations are completed.

16. A data center system, comprising: Multiple application servers; as well as Multiple storage servers, Each storage server includes a near memory processing (PNM) device, wherein the PNM device is configured to execute the search method according to any one of claims 6 to 11.

17. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, the search method according to any one of claims 1 to 11 is implemented.

Images