Retrieval method, host and near memory processing device

Abstract

The invention provides a retrieval method, a host and near-memory processing equipment. The retrieval method comprises the following steps: receiving a query vector; starting a plurality of pieces of near memory processing (PNM) equipment to execute sub-retrieval corresponding to the query vector in data subsets stored in the equipment respectively; receiving a plurality of sub-results returned by the plurality of PNM devices based on the sub-retrieval; and outputting a retrieval result about the query vector based on the plurality of sub-results.

Description

Technical Field

[0001] The present disclosure relates to the field of data retrieval, and in particular to a retrieval method, a host, and a near memory processing device. Background Art

[0002] Recently, data retrieval techniques, such as dense retrieval, have played a significant role in supporting various machine learning applications, including search engines, data mining, databases, and recommender systems. For example, dense retrieval methods involve searching a dataset for objects similar to the query. For example, if a user browses to a picture of a dog, similar images can be found in the dataset.

[0003] However, in dense retrieval, a large amount of data transfer between the central processing unit (CPU) and the memory leads to low retrieval performance, and a large amount of calculation in dense retrieval leads to high CPU load. Summary of the Invention

[0004] The present disclosure provides a retrieval method, a host, and a near memory processing device to solve part or all of the above problems.

[0005] According to one aspect of the present disclosure, a retrieval method is provided, which is executed by a host device, and the method includes: receiving a query vector; initiating multiple near memory processing (PNM) devices to perform a sub-search corresponding to the query vector; receiving multiple sub-results, each of the multiple sub-results being provided by one of the multiple PNM devices based on the sub-search performed in a data subset stored in the respective PNM devices; and outputting a retrieval result corresponding to the query vector based on the multiple sub-results.

[0006] Optionally, starting the multiple PNM devices to perform sub-searches corresponding to the query vector includes: providing retrieval parameters to the multiple PNM devices to start the multiple PNM devices to perform the sub-searches in parallel in the data subsets stored in their respective PNM devices, wherein the retrieval parameters include: parameters related to the query vector and the return address of the sub-results.

[0007] Optionally, the multiple PNM devices are all or part of the PNM devices in a PNM device pool.

[0008] Optionally, outputting the search result includes: merging the multiple sub-results to output the search result.

[0009] Optionally, the sub-search is performed based on an approximate nearest neighbor search (ANNS) method, wherein the data subsets stored in the respective PNM devices are subgraphs comprising a subset of data vectors, wherein the subgraphs are obtained by splitting a graph comprising all data vectors used for the ANNS method.

[0010] Optionally, the retrieval parameters also include a candidate capacity of the ANNS method, wherein the candidate capacity indicates the number of candidate data vectors in the candidate list and the candidate capacity is less than a reference number of results, wherein outputting the retrieval result includes: sorting the multiple sub-results based on the distance between the multiple sub-results and the query vector, and outputting the reference number of sorted sub-results as the retrieval result.

[0011] According to another aspect of the present disclosure, a retrieval method is provided, which is performed by a near memory processing (PNM) device, and the method includes: receiving a request to perform a sub-retrieval corresponding to a query vector; performing the sub-retrieval in a data subset stored in the PNM device; and sending a sub-result based on the sub-retrieval to a host device.

[0012] Optionally, executing the sub-search includes: executing the sub-search in the data subset according to search parameters provided by the host device to start the sub-search, wherein the search parameters include: parameters related to the query vector and a return address of the sub-result.

[0013] Optionally, the sub-search is performed based on an approximate nearest neighbor search (ANNS) method, and the data subset is a subgraph including a subset of data vectors, and wherein the subgraph is obtained by splitting a graph including all data vectors used for the ANNS method.

[0014] Optionally, the retrieval parameters also include a candidate capacity of the ANNS method, wherein the candidate capacity indicates the number of candidate data vectors in the candidate list, and wherein, performing the sub-search in the data subset according to the retrieval parameters includes: searching the subgraph for a first number of data vectors that are closest to the query vector according to the retrieval parameters, wherein the first number of data vectors satisfies the candidate capacity.

[0015] Optionally, searching the subgraph for a first number of data vectors closest to the query vector includes: selecting a starting data vector in the subgraph as a current vector, and initializing a candidate list to include the starting data vector; traversing the neighbor data vectors of the current vector in the subgraph; calculating the distance between the neighbor data vector and the query vector; updating the candidate list based on a comparison of the distances; selecting the candidate data vector in the updated candidate list that is closest to the query vector as the current vector, and repeating the operation of traversing the neighbor data vectors of the current vector in the subgraph based on changes in the current vector.

[0016] Optionally, based on the comparison of the distances, updating the candidate list includes: selecting a first number of data vectors closest to the query vector from the neighbor data vectors and the existing candidate data vectors as new candidate data vectors; and updating the candidate list based on the new candidate data vectors including new candidate data vectors that are different from the existing candidate data vectors.

[0017] Optionally, sending a sub-result based on the sub-search to the host device includes: sending a candidate data vector in a candidate list as the sub-result to the host device according to a return address of the sub-result.

[0018] According to another aspect of the present disclosure, a host device is provided, comprising: a memory storing one or more instructions; and a processor configured to execute the one or more instructions to perform the following operations: receiving a query vector; initiating multiple near memory processing (PNM) devices to perform a sub-search corresponding to the query vector; receiving multiple sub-results, each of the multiple sub-results being provided by one of the multiple PNM devices based on the sub-search performed in a data subset stored in the respective PNM devices; and outputting a search result corresponding to the query vector based on the multiple sub-results.

[0019] Optionally, the processor is further configured to perform the following operations: provide retrieval parameters to the multiple PNM devices to enable the multiple PNM devices to perform the sub-search in parallel in the data subsets stored in their respective PNM devices, wherein the retrieval parameters include: parameters related to the query vector and the return address of the sub-result.

[0020] Optionally, the multiple PNM devices are all or part of the PNM devices in a PNM device pool.

[0021] Optionally, the processor is further configured to perform the following operation: merge the multiple sub-results to output the search result.

[0022] Optionally, the sub-search is performed based on an approximate nearest neighbor search (ANNS) method, and wherein the stored data subset is a subgraph including a subset of data vectors, wherein the subgraph is obtained by splitting a graph including all data vectors used for the ANNS method.

[0023] Optionally, the retrieval parameters also include a candidate capacity of the ANNS method, wherein the candidate capacity indicates the number of candidate data vectors in the candidate list and the candidate capacity is less than a reference number of results, wherein the processor is further configured to perform the following operations: sort the multiple sub-results based on the distance between the multiple sub-results and the query vector, and output the reference number of sorted sub-results as the retrieval results.

[0024] According to another aspect of the present disclosure, a near memory processing (PNM) device is provided, comprising: a memory storing a data subset; and a processor configured to: receive a request to perform a sub-search corresponding to a query vector; perform the sub-search in the data subset stored in the PNM device; and send a sub-result based on the sub-search to a host device.

[0025] Optionally, the processor is further configured to: perform the sub-search in the data subset according to search parameters for starting the sub-search provided by the host device, wherein the search parameters include: parameters related to the query vector and a return address of the sub-result.

[0026] Optionally, the sub-search is performed based on an approximate nearest neighbor search (ANNS) method, and the data subset is a subgraph including a subset of data vectors, and wherein the subgraph is obtained by splitting a graph including all data vectors used for the ANNS method.

[0027] Optionally, the retrieval parameters also include a candidate capacity of the ANNS method, wherein the candidate capacity indicates the number of candidate data vectors in the candidate list, and wherein the processor is further configured to: search the subgraph for a first number of data vectors that are closest to the query vector according to the retrieval parameters, wherein the first number of data vectors satisfies the candidate capacity.

[0028] Optionally, the processor is further configured to: select a starting data vector as the current vector in the subgraph, and initialize a candidate list including the starting data vector; traverse the neighbor data vectors of the current vector in the subgraph; calculate the distance between the neighbor data vector and the query vector; update the candidate list based on the comparison of the distance; select the candidate data vector in the updated candidate list that is closest to the query vector as the current vector, and repeat the operation of traversing the neighbor data vectors of the current vector in the subgraph based on the change of the current vector.

[0029] Optionally, the processor is further configured to: select a first number of data vectors that are closest to the query vector from the neighbor data vectors and the existing candidate data vectors as new candidate data vectors; and update the candidate list based on the new candidate data vectors including new candidate data vectors that are different from the existing candidate data vectors.

[0030] Optionally, the processor is further configured to: send a candidate data vector in a candidate list to the host device as a sub-result according to a return address of the sub-result.

[0031] According to another aspect of the present disclosure, a system is provided, comprising: a main processor; a memory; a plurality of near memory processing (PNM) devices; and a storage device, wherein each of the plurality of PNM devices is configured to execute a retrieval method, the retrieval method comprising: receiving a request to execute a sub-search corresponding to a query vector; executing the sub-search in a subset of data stored in the PNM device; and sending a sub-result based on the sub-search to a host device.

[0032] According to another aspect of the present disclosure, a storage system is provided, comprising: a host device; and a plurality of near memory processing (PNM) devices. The host device is configured to: receive a query vector; enable the plurality of PNM devices to perform a sub-search corresponding to the query vector; receive a plurality of sub-results, each of the plurality of sub-results being provided by one of the plurality of PNM devices based on the sub-search; output a search result corresponding to the query vector based on the plurality of sub-results; and each of the plurality of PNM devices is configured to: perform the sub-search on a data subset stored in the respective PNM device; and transmit the sub-results based on the sub-search to the host device.

[0033] According to another aspect of the present disclosure, a data center system is provided, comprising: a plurality of application servers; and a plurality of storage servers. Each application server includes a plurality of near memory processing (PNM) devices, wherein each of the plurality of PNM devices is configured to execute a search method, the search method comprising: receiving a request to execute a sub-search corresponding to a query vector; executing the sub-search in a data subset stored in the PNM device; and transmitting a sub-result based on the sub-search to a host device.

[0034] According to another aspect of the present disclosure, a computer-readable storage medium storing a computer program is provided, which, when executed by a processor, is configured to perform a retrieval method, the retrieval method comprising: receiving a query vector; initiating multiple near memory processing (PNM) devices to perform a sub-search corresponding to the query vector; receiving multiple sub-results, each of the multiple sub-results being provided by one of the multiple PNM devices based on the sub-search performed in a data subset stored in the respective PNM devices; and outputting a retrieval result corresponding to the query vector based on the multiple sub-results.

[0035] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0037] Figure 1A and Figure 1B Schematic diagram showing the comparison between the KNN algorithm and the ANNS algorithm.

[0038] Figure 2 Shows a schematic diagram of the CXL-ANNS hardware structure.

[0039] Figure 3 Shows a schematic diagram of the CXL-ANNS software structure.

[0040] Figure 4 A schematic diagram showing the calculations in the CPU of CXL-ANNS.

[0041] Figure 5 A structural schematic diagram according to an embodiment of the present disclosure is shown.

[0042] Figure 6 A flowchart illustrating a retrieval method applied to a host according to an embodiment of the present disclosure is shown.

[0043] Figure 7 A schematic diagram illustrating communication between a retrieval agent module in a host and a PNM device according to an embodiment of the present disclosure is shown.

[0044] Figure 8 A flowchart of a retrieval agent algorithm according to an embodiment of the present disclosure is shown.

[0045] Figure 9A and Figure 9B A schematic diagram illustrating segmented dense retrieval according to an embodiment of the present disclosure is shown.

[0046] Figure 10 A flowchart of a retrieval method applied to a PNM device according to an embodiment of the present disclosure is shown.

[0047] Figure 11 A flowchart of ANNS-based retrieval according to an embodiment of the present disclosure is shown.

[0048] Figure 12 The figure shows an overall data transmission flow chart of the retrieval method according to an embodiment of the present disclosure.

[0049] Figure 13 A schematic diagram illustrating a host according to an embodiment of the present disclosure.

[0050] Figure 14 A schematic diagram illustrating a PNM device according to an embodiment of the present disclosure is shown.

[0051] Figure 15 is a schematic diagram of a system to which a storage device is applied according to an embodiment of the present disclosure.

[0052] Figure 16is a block diagram of a host storage system according to an embodiment of the present disclosure.

[0053] Figure 17 is a diagram of a data center to which a storage device is applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0054] In order to promote the understanding of the principles of the present disclosure, reference will now be made to various embodiments and specific language will be used to describe them. The embodiments herein and their various features and advantageous details will be more fully explained with reference to the non-limiting embodiments shown in the accompanying drawings and described in detail in the following description. Descriptions of well-known components and processing technologies are omitted so as not to unnecessarily obscure the embodiments herein. In addition, the various embodiments described herein are not necessarily mutually exclusive, as certain embodiments can be combined with one or more other embodiments to form new embodiments. Unless otherwise indicated, the term "or" used herein refers to a non-exclusive or. The examples used herein are only for the purpose of facilitating the understanding of the manner in which the embodiments herein are implemented and to further enable those skilled in the art to implement the embodiments herein. Therefore, these examples should not be understood as limiting the scope of the embodiments herein.

[0055] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0056] It should be noted that the phrase "at least one of the items" in this disclosure includes three types of parallel situations: "any one of the items", "a combination of any multiple items of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. For another example, "performing at least one of operation 1 and operation 2" means the following three parallel situations: (1) performing operation 1; (2) performing operation 2; and (3) performing operation 1 and operation 2.

[0057] Reference throughout this specification to "on one hand," "on the other hand," or similar language means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases "in an embodiment," "in another embodiment," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0058] The terms "comprises," "includes," "has," "having" or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process or method that comprises a series of steps does not include only those steps but may also include other steps not expressly listed or inherent to the process or method. Similarly, one or more devices or subsystems or elements or structures or components followed by "comprising..." do not, without more limitations, preclude the presence of other devices or other subsystems or other elements or other structures or other components or additional devices or additional subsystems or additional elements or additional structures or additional components.

[0059] As is conventional in the art, embodiments may be described and illustrated using blocks that implement one or more functions. These blocks, which may be referred to herein as managers, units, modules, hardware components, etc., are implemented by analog and / or digital circuits (such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hard-wired circuits, etc.) and may be selectively driven by firmware. For example, the circuits may be embodied in one or more semiconductor chips or on a substrate support (such as a printed circuit board). The circuits that make up the blocks may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware that performs certain functions of the block and a processor that performs other functions of the block. Without departing from the scope of this disclosure, each block in this embodiment may be physically separated into two or more interacting discrete blocks. Similarly, without departing from the scope of this disclosure, the blocks in the embodiments may also be physically combined into more complex blocks. However, the disclosure is not limited thereto, and therefore, the blocks that may be referred to herein as managers, units, modules, etc. may be software modules implemented by software code, program code, software instructions, etc. The software modules may be executed on a processor.

[0060] In retrieval technology, various algorithms can be implemented to perform intensive retrieval. For example, the K-Nearest Neighbor (KNN) algorithm or the Approximate Nearest Neighbor Search (ANNS) algorithm can be implemented to perform intensive retrieval. However, the present disclosure is not limited thereto. Intensive retrieval uses a feature vector containing hundreds of dimensions to represent an object. Furthermore, intensive retrieval uses the distance between feature vectors to represent the similarity between multiple objects.

[0061] The KNN algorithm compares the distance between data vectors (feature vectors of objects in a dataset, hereinafter referred to as vectors) and query vectors (feature vectors of the query object) to obtain the K most similar results. Due to the large number of dimensions corresponding to the feature vectors and the large amount of data to be retrieved, KNN requires a large amount of data computation. For example, in an e-commerce platform in the related field, intensive retrieval recommends 100 neighbors from 2 billion vectors (128 dimensions) for a single query.

[0062] The ANNS algorithm can use a graph to represent the relationship between each object (among multiple objects) and its nearest neighbors. The ANNS algorithm limits the search vector to only a subset of neighbors that are likely to be the nearest neighbors, thereby reducing the amount of computation.

[0063] Figure 1A and Figure 1B A diagram showing a comparison between the KNN algorithm and the ANNS algorithm. For example, Figure 1A A diagram showing the KNN algorithm and Figure 1B Schematic diagram of ANNS algorithm is shown. Figure 1A As shown in , the KNN algorithm needs to calculate the distance from the query vector to the data vector (A vector to G vector) 7 times in total. Figure 1B As shown in the figure, according to the ANNS algorithm, if vector A is used as the starting data vector, the neighbor vectors B and C, F, G, and E are obtained in sequence. Ultimately, only five distances between the query vector and the data vector need to be calculated. For example, ① vectors B and C are neighbors of vector A, and vector B is closer to the query vector; ② vector F is a neighbor of vector B and is closer to the query vector; ③ vector G is a neighbor of vector F and is closer to the query vector; ④ vector E is a neighbor of vector G but is farther from the query vector than vector G. While using a graph for the ANNS algorithm can reduce computational complexity, it requires more memory to cache the graph and data vectors.

[0064] Compute Express Link (CXL) Approximate Nearest Neighbor Search (CXL-ANNS) extends memory capacity by using the CXL memory pool. Figure 2 A schematic diagram of the CXL-ANNS hardware architecture is shown. The hardware architecture includes two components: a host computer and a CXL memory pool. The host computer includes a CPU and memory, and the memory pool includes CXL memory expanders connected to a CXL switch based on the CXL protocol. Figure 3 The following is a schematic diagram of the CXL-ANNS software structure. Figure 3, the operations implemented in the CPU include: graph traversal, candidate update and distance calculation. For example, graph traversal can be used to implement traversal of the graph in the memory and the CXL pool to obtain the neighbor data vectors of the current vector. For example, candidate update can be performed by comparing the distance between the data vector in the candidate (e.g., candidate list) and the query vector, sorting the data vector in ascending order of distance, and retaining only K results. For example, distance calculation may include calculating the distance between two vectors. The inner graph and candidates are stored in the memory, and the candidates are used to save the neighbor data vectors obtained by the graph traversal and the distance between the neighbor data vector and the query vector. The outer graph and data vectors are stored in the CXL pool. In order to improve performance, CXL-ANNS divides the graph into an inner graph and an outer graph, and the inner graph with high access frequency is cached in the memory (e.g., the memory may include but is not limited to dynamic random access memory (DRAM)).

[0065] However, the large amount of data transfer in CXL-ANNS leads to poor performance. In CXL-ANNS, graph traversal, distance calculation, and candidate updates require large amounts of data transfer between the CPU and memory. Although CXL-ANNS stores frequently used inner graphs in host memory, it still needs to retrieve outer graphs and data vectors from the CXL pool. Furthermore, the latency of the CXL memory extender is higher than that of host memory, and data transfer at high latency degrades retrieval performance.

[0066] In addition, CXL-ANNS has a large number of computations that result in high CPU load. For example, since feature vectors contain hundreds of dimensions, distance calculations between vectors result in high CPU load. Figure 4 This figure shows the calculation diagram of the CPU of CXL-ANNS. Figure 4 Although CXL-ANNS splits the data and query vectors into multiple sub-vectors and calculates sub-distances in the CXL memory extender, the accumulation of sub-distances still requires CPU calculation. Furthermore, when updating candidates, the distances between the query vector and new neighbor data vectors (e.g., adjacent vectors) discovered through graph traversal must be compared and sorted with the distances between the candidate data vectors and the query vector, which also results in high CPU load.

[0067] In order to solve the problems in CXL-ANNS, one or more embodiments of the present disclosure provide a retrieval method that utilizes Process Near Memory (PNM) technology. In PNM technology, memory and logic chips are integrated into advanced integrated circuit packages to reduce data movement between the CPU and memory. For example, PNM technology uses memory integrated into advanced integrated circuit packages to perform data calculations. In the near-memory processing retrieval method, computing functions can be performed closer to the memory, which can reduce bottlenecks in data transmission between the CPU and the memory. PNM technology based on the CXL interface has been proven to be an excellent solution for large-capacity artificial intelligence (AI) model processing. For example, in tests performed using the PNM method based on the CXL interface, the results showed that the performance in applications such as recommendation systems or in-memory databases that require high memory bandwidth was more than doubled compared to related technical methods.

[0068] According to an embodiment of the present disclosure, memory capacity can be expanded through one or more PNM devices. For example, a dense search can be split (or divided) into multiple sub-searches and offloaded to one or more PNM devices. This reduces the CPU load and the data transmission between the CPU and the memory. Figures 5 to 17 The retrieval method, host, and PNM device according to the embodiments of the present disclosure are described in detail.

[0069] Figure 5 A structural schematic diagram according to an embodiment of the present disclosure is shown.

[0070] Reference Figure 5, the structure may include a host and a PNM device pool. The host may include a CPU and a memory, and the PNM device pool may include multiple PNM devices. The host may also be referred to as a host device. In ANNS, all data vectors are stored in one graph, and the high-density graph results in slow graph traversal. According to an embodiment of the present disclosure, the graph may be split into multiple subgraphs. For example, multiple subgraphs may be stored (or saved) in PNM devices, and each PNM device may traverse the subgraph stored in its respective PNM device. According to an embodiment, each PNM device may traverse the subgraphs stored therein in parallel. For example, a first PNM device may traverse a first subgraph stored in a first PNM device, and a second PNM device may traverse a second subgraph stored in a second PNM device. The first PNM device and the second PNM device may traverse their respective subgraphs in parallel. In this way, the density of the graph and the number of data vectors to be traversed by each PNM device may be reduced. In addition, parallel retrieval of multiple PNM devices may improve retrieval performance. According to an embodiment, the graph may be split into a number of subgraphs equal to the number of PNM devices. However, the present disclosure is not limited thereto, and therefore, the number of subgraphs may be different from the number of PNM devices. For example, the number of subgraphs may be smaller than the number of PNM devices.

[0071] According to an embodiment, on the host side, the operations of two modules may be implemented in the CPU. For example, the CPU may be configured to implement a retrieval agent module and a merge result module. However, the present disclosure is not limited thereto, and therefore, the CPU may be configured to implement other modules and / or perform other operations. For example, the retrieval agent module and the merge result module may be software modules executed by the CPU. For example, the software module may be stored in a storage or memory. According to an embodiment, the retrieval agent module may be responsible for communicating with one or more PNM devices through an interface and providing the necessary parameters for retrieval in the PNM device. The interface may refer to pnm_retrieval. According to an embodiment, the merge result module may sort or modify the sub-results retrieved by one or more PNM devices according to system requirements to obtain the final result. According to an embodiment, the memory may store the retrieval sub-results obtained from one or more PNM devices.

[0072] According to an embodiment, on the PNM device pool side, each of the one or more PNM devices may include a PNM computing unit and a memory. According to an embodiment, the PNM computing unit may be a PNM processor. However, the present disclosure is not limited thereto, and therefore, the PNM computing unit may be implemented as a logic circuit. For example, the operation of the retrieval core module may be implemented in the PNM processor. For example, the retrieval core module may be a software module executed by the PNM processor, and the retrieval core module performs a sub-retrieval in the PNM device according to the parameters of the interface (e.g., pnm_retrieval). A subvector is a subset of a data vector allocated to a PNM device, and a subgraph (e.g., subgraph data) is a graph allocated to a PNM device. For example, the subgraph data may represent the relative relationship (e.g., connection relationship) of a subset of a data vector in a graph. According to an embodiment, the memory of the PNM device may store subgraphs and subvectors.

[0073] It should be understood that the above structures are merely examples and the present disclosure is not limited thereto.

[0074] Figure 6 A flowchart illustrating a retrieval method applied to a host according to an embodiment of the present disclosure is shown.

[0075] In operation S610, the method may include receiving a query vector. For example, the host may receive the query vector. According to an embodiment, the host may receive the query vector input by the user and start a search process corresponding to the query vector.

[0076] In operation S620, the method may include initiating multiple near memory processing PNM devices to perform sub-searches corresponding to the query vector. For example, the host may perform a control operation to initiate multiple PNM devices to perform sub-searches corresponding to the query vector. For example, the sub-search may be performed on a data subset stored separately in each of the multiple PNM devices. For example, a first sub-search may be performed on a first data subset stored in a first PNM device among the multiple PNM devices, and a second sub-search may be performed on a second data subset stored in a second PNM device among the multiple PNM devices. According to an embodiment of the present disclosure, controlling the initiation of multiple PNM devices to perform sub-searches regarding the query vector may include: providing search parameters to the multiple PNM devices to initiate the multiple PNM devices to perform sub-searches in parallel in the separately stored data subsets. For example, the search parameters include, but are not limited to, parameters related to the query vector and return addresses of sub-results.

[0077] Figure 7 FIG. 1 is a schematic diagram showing the communication between the retrieval agent module in the host and the PNM device according to an embodiment of the present disclosure. Figure 7, the retrieval agent module running on the host side may be responsible for communicating with the PNM device and providing parameters (e.g., retrieval parameters) for performing retrieval (e.g., sub-retrieval) in the PNM device. The retrieval agent module may initiate a sub-retrieval for each PNM device through, for example, the interface "pnm_retrieval". According to an embodiment, when the sub-retrieval in the PNM device is completed, the sub-result is stored in the address indicated by the parameter "result". The interface "pnm_retrieval" is defined as follows: int pnm_retrieval(*query_vector, query_size, **result, index, count). Table 1 shows the meaning of each parameter as follows.

[0078] Table 1

[0079] Parameter Type Parameter name describe uint64_t* query_vector Query vector address uint64_t query_size Query vector size uint64_t* result Sub-result storage address uint32_t index PNM Device Index uint16_t count Candidate capacity

[0080] The parameters may include, but are not limited to, parameters related to the query vector and the sub-result storage address. For example, the parameters related to the query vector include, but are not limited to, the query vector address and the query vector size. For example, the sub-result storage address may be the return address of the sub-result. According to an embodiment, the sub-result storage address may be an address in the host memory. After the sub-search is completed, the PNM device may return the sub-result to the host memory according to the sub-result storage address.

[0081] According to an embodiment of the present disclosure, the plurality of PNM devices may be all PNM devices in a PNM device pool. However, the present disclosure is not limited thereto, and therefore, the plurality of PNM devices may be part of the PNM devices in the PNM device pool.

[0082] For example, a PNM device pool may include multiple PNM devices, and the host may control and start all or part of the PNM devices in the PNM device pool.

[0083] Figure 8 FIG. 1 shows a flowchart of a search agent algorithm according to an embodiment of the present disclosure. Figure 8In operation S810, the search agent module activates all PNM devices in the PNM device pool by default to perform a sub-search. In operation S810, the method may include obtaining interface-related parameters. For example, the search agent module may obtain interface-related parameters. For example, interface-related parameters may include, but are not limited to, query vector address, query vector size, sub-result storage address, PNM device index, and candidate capacity. In operation S820, the method may include determining whether all PNM devices are activated. For example, the search agent module may determine whether all PNM devices are activated. In the example case where the result of the determination is "yes", the process ends. In the example case where the result of the determination is "no", the process proceeds to operation S830. In operation S830, the method includes activating PNM devices to perform sub-searches until all PNM devices are activated. For example, the search agent module may use the interface to activate PNM devices to perform sub-searches until all PNM devices are activated.

[0084] According to an embodiment, the retrieval agent module may only activate some of the PNM devices in the PNM device pool. For example, the PNM device pool may include four PNM devices, namely, PNM device A, PNM device B, PNM device C, and PNM device D. According to an embodiment, the host may control the activation of any two of the PNM devices, such as PNM device A and PNM device C, and have these two PNM devices return two sub-results. For example, the retrieval agent module in the host may activate PNM device A and PNM device C through the "pnm_retrieval" interface. The PNM device index "index" in the "pnm_retrieval" interface for activating PNM device A may be 1, while the "index" in the "pnm_retrieval" interface for activating PNM device C may be 2. Since activating two PNM devices returns two sub-results, the result includes the positions of the two sub-results. PNM device A may return the sub-result at the first sub-result position in the result according to the "index" value of 1, while PNM device C may return the sub-result at the second sub-result position in the result according to the "index" value of 2.

[0085] Return to reference Figure 6 In operation S630, the method may include receiving a plurality of sub-results. For example, the host may receive a plurality of sub-results returned by a plurality of PNM devices based on sub-searches performed by the respective PNM devices.

[0086] In operation S640, the method may include outputting a search result. For example, the host may output a search result corresponding to the query vector based on a plurality of sub-results.

[0087] According to an embodiment of the present disclosure, outputting a search result related to a query vector based on a plurality of sub-results may include: merging the plurality of sub-results to output the search result.

[0088] According to an embodiment of the present disclosure, multiple PNM devices can return sub-results based on their own sub-searches to a host, and the number of sub-results is consistent with the number of enabled PNM devices. The host receives multiple sub-results stored in the host memory and, through a result merging module, can merge the multiple sub-results to output a search result. Here, merging can include sorting or modifying the multiple sub-results. For example, the multiple sub-results (e.g., data vectors within the sub-results) can be sorted by similarity (e.g., distance). For example, only the search results most similar to the query vector can be output. For example, only a predetermined number of search results most similar to the query vector can be output. According to an embodiment, search results whose similarity to the query vector is less than or equal to a reference value can be selected as output. For example, search results whose distance to the query vector is less than or equal to a reference value can be selected as output. According to an embodiment, the method can include removing sub-results (e.g., data vectors within the sub-results) that do not meet a specific condition, sorting the remaining sub-results (e.g., sorting by similarity), and outputting a reference number of results. Here, the reference number of results can be a predetermined number of results. The predetermined number of results may be, for example, a number input by a user or a number of final results required by a system, however, the present disclosure is not limited thereto.

[0089] According to the search method of an embodiment of the present disclosure, a search (e.g., a dense search) is divided into multiple sub-searches and offloaded to multiple PNM devices, thereby reducing the CPU load and data transmission between the CPU and memory. In addition, multiple PNM devices can search in parallel, thereby improving search performance.

[0090] According to an embodiment of the present disclosure, sub-searching can be performed based on approximate nearest neighbor search (ANNS), and the stored data subset can be a subgraph including a subset of data vectors. For example, the subgraph can be obtained by splitting the graph including all data vectors used for ANNS.

[0091] It should be understood that the retrieval is described by taking the retrieval based on ANNS as an example, but the present disclosure is not limited thereto, and the retrieval may be any intensive retrieval that can be divided or split.

[0092] Figure 9A and Figure 9B A schematic diagram of segmented dense retrieval according to an embodiment of the present disclosure is shown. For example, Figure 9A shows the diagram corresponding to CXL-ANNS, and Figure 9B A diagram corresponding to a search method according to an embodiment of the present disclosure is shown. Figure 9AA diagram including all data vectors (including vectors A to G) for ANNS is shown, and Figure 9B The graph is split into subgraphs 1 and 2, which respectively include data vector subset 1 (including vector A, vector C, vector E and vector G) and data vector subset 2 (including vector B, vector D and vector F). Figure 2 The subgraphs are stored in the memory of the PNM devices in the form of subgraph data and subvector data. For example, subgraph 1 may be stored in the first PNM device 1 and subgraph 2 may be stored in the memory of the PNM device 2. Figure 2 may be stored in the second PNM device 2. According to an embodiment, the sub-vector data may represent a data vector subset, and the sub-graph data may represent a relative position (eg, a connection relationship) of the data vector subset in the sub-vector data.

[0093] In CXL-ANNS, the graph traversal is performed four times. The four graph traversals may include: traversing vectors B and C with vector A as the current vector (for example, ①), traversing vector F with vector B as the current vector (for example, ②), traversing vector G with vector F as the current vector (for example, ③), and traversing vector E with vector G as the current vector (for example, ④). In the retrieval method according to an embodiment of the present disclosure, the graph is split into two subgraphs (subgraph 1 and subgraph 2). Figure 2 ) and offloaded to the PNM devices (e.g., PNM device 1 and PNM device 2) for sub-search. In this case, three graph traversals can be performed in PNM device 1, and one graph traversal can be performed in PNM device 2. Since PNM device 1 and PNM device 2 perform sub-searches in parallel, the search method according to the embodiment of the present disclosure performs at most three graph traversals. For example, the three graph traversals performed in PNM device 1 may include: traversing vector C with vector A as the current vector (e.g., ①), traversing vector E with vector C as the current vector (e.g., ②), and traversing vector G with vector E as the current vector (e.g., ③), and the one graph traversal performed in PNM device 2 may include: traversing vector D and vector F with vector B as the current vector (e.g., ①).

[0094] According to an embodiment of the present disclosure, the search parameters may further include a candidate capacity of the ANNS. For example, the candidate capacity may indicate the number of candidate data vectors in the candidate and the candidate capacity may be less than a predetermined number of results (i.e., a predetermined number of results). For example, outputting search results for the query vector based on multiple sub-results may include, but is not limited to, sorting the multiple sub-results based on the distance between the multiple sub-results and the query vector, and outputting the predetermined number of sorted sub-results as the search results.

[0095] According to an embodiment of the present disclosure, as shown in Table 1, the parameters passed by the interface "pnm_retrieval" may further include a candidate capacity "count" (e.g., the value is N, and N is a positive integer). The candidate capacity indicates the number of candidate data vectors in a candidate (e.g., a candidate list). After the sub-retrieval is completed, the PNM device returns the candidate data vectors in the candidate as sub-results. For example, the sub-results returned by the PNM device include the candidate capacity number (e.g., N) of data vectors (e.g., candidate data vectors). For example, in the case of starting M (M is a positive integer) PNM devices for sub-retrieval, M sub-results are returned, and each sub-result includes N data vectors. According to an embodiment of the present disclosure, after receiving multiple sub-results, the host further sorts the multiple sub-results based on the distances between the multiple sub-results and the query vector, and outputs the sorted sub-results with a predetermined result number (e.g., P, and P is a positive integer) as the retrieval result. For example, in the case of starting M PNM devices for sub-retrieval, the host receives M*N data vectors stored in the host memory returned by the PNM device. The host (e.g., the merge result module) can calculate the distances between the M*N data vectors and the query vector respectively, and then based on the sorting result (e.g., ascending order) of the M*N distances, output the top P sorted data vectors as the retrieval result. According to an embodiment, the sub-results returned by the PNM to the host include the data vectors and the distances from the data vectors to the query vector. For example, the host receives the M*N data vectors stored in the host memory returned by the PNM device and the distances from the M*N data vectors to the query vector, and then based on the sorting result (e.g., ascending order) of the M*N distances, outputs the top P sorted data vectors as the retrieval result.

[0096] According to an embodiment of the present disclosure, the candidate capacity may be less than the predetermined result number, that is, N<P. The candidate capacity being less than the predetermined result number (e.g., reduction of the candidate capacity) can reduce the computational amount of the retrieval (e.g., sub-retrieval). According to an embodiment, in order to satisfy the final output of P retrieval results, N may satisfy that the product of N and M is greater than or equal to P.

[0097] For the retrieval method according to an embodiment of the present disclosure as described above, first, a sub-graph including a subset of data vectors is stored in the PNM device memory and the sub-retrieval is completed in the PNM device, reducing the data transmission between the CPU and the memory during the retrieval; second, the sub-retrieval is offloaded to the PNM device, and the host only starts the sub-retrieval in the PNM device through the interface and performs the merging of a small number of sub-results, reducing the load of the CPU; third, the retrieval is divided into multiple sub-retrievals that are executed in parallel in the PNM device, and the reduction of the graph density and the reduction of the candidate capacity reduce the computational amount of the retrieval and improve the performance of the retrieval.

[0098] Figure 10A flowchart of a retrieval method applied to a PNM device according to an embodiment of the present disclosure is shown.

[0099] In operation S1010, the method may include performing a sub-search corresponding to the query vector. For example, the PNM device may perform a sub-search corresponding to the query vector based on a stored data subset.

[0100] According to an embodiment of the present disclosure, performing a sub-search on a query vector in a stored data subset may include performing the sub-search on the stored data subset based on search parameters provided by a host to initiate the sub-search. For example, the search parameters include, but are not limited to, parameters related to the query vector and a return address for the sub-result.

[0101] According to an embodiment of the present disclosure, a retrieval agent module on the host side can initiate a sub-retrieval for each PNM device through an interface (e.g., pnm_retrieval). For example, the PNM device can receive a request to perform a sub-retrieval on a subset of data stored in the PNM device. The retrieval core module in the PNM device performs a sub-retrieval on the subset of data stored in the memory based on parameters (e.g., retrieval parameters) provided by the host through the interface. As shown in Table 1, the retrieval parameters may include parameters related to the query vector (e.g., including the query vector address and query vector size) and the sub-result storage address (e.g., the return address of the sub-result).

[0102] According to an embodiment of the present disclosure, sub-searching can be performed based on approximate nearest neighbor search (ANNS), and the stored data subset can be a subgraph including a subset of data vectors. For example, the subgraph can be obtained by splitting the graph including all data vectors used for ANNS.

[0103] It should be understood that the retrieval is described by taking the retrieval based on ANNS as an example, but the present disclosure is not limited thereto, and the retrieval may be any intensive retrieval that can be divided.

[0104] According to an embodiment of the present disclosure, a graph including all data vectors for ANNS may be pre-split into multiple subgraphs and stored in memories of multiple PNM devices, for example, a subgraph including a subset of data vectors.

[0105] According to an embodiment of the present disclosure, the search parameters may further include a candidate capacity of the ANNS. For example, the candidate capacity may indicate the number of candidate data vectors in the candidate. For example, performing a sub-search in a stored data subset based on the search parameters may include searching the subgraph for a first number of data vectors closest to the query vector based on the search parameters, where the first number of data vectors satisfies the candidate capacity.

[0106] According to an embodiment of the present disclosure, as shown in Table 1, the retrieval parameters may further include a candidate capacity "count" related to ANNS. For example, the candidate capacity may indicate the number of candidate data vectors in the candidate. However, the present disclosure is not limited thereto, and therefore, the candidate capacity may limit the number of candidate data vectors in the candidate. Here, the first number of data vectors satisfying the candidate capacity may be, for example, but not limited to, the first number is less than or equal to the candidate capacity. The candidate capacity may be less than the reference number of the final result, which reduces the comparison in the candidate update. Here, the reference number of the final result may be a predetermined number of the final result. According to an embodiment, a count value of 1 may indicate that the candidate is cancelled and only one neighbor data vector closest to the query vector is recorded.

[0107] The retrieval core module in the PNM device can search for candidate capacity data vectors closest to the query vector in the subgraph based on ANNS according to parameters related to the query vector (eg, the address of the query vector and the size of the query vector).

[0108] According to an embodiment of the present disclosure, searching for a candidate data vector closest to a query vector in a subgraph may include, but is not limited to, selecting a starting data vector in the subgraph as a current vector, and initializing the candidates to include the starting data vector; traversing the neighbor data vectors of the current vector in the subgraph; calculating the distance between the neighbor data vector and the query vector; updating the candidates based on a comparison of the distances; selecting the candidate data vector closest to the query vector from the updated candidates as the current vector, and, in the event that the current vector is changed, returning to the step of traversing the neighbor data vectors of the current vector in the subgraph.

[0109] According to an embodiment of the present disclosure, updating candidates based on distance comparison may include: selecting a candidate data vector closest to the query vector among neighbor data vectors and existing candidate data vectors as a new candidate data vector; and updating the candidate when there is a new candidate data vector that is different from the existing candidate data vector.

[0110] Figure 11 A flowchart based on ANNS retrieval according to an embodiment of the present disclosure is shown. For example, Figure 11 A method of searching in a graph including all data vectors is shown as an example, such as in Figure 1B However, the present disclosure is not limited thereto, and therefore, Figure 11 The search method shown in FIG can be performed on a search that is divided into multiple sub-searches, which are offloaded to the PNM device, and the search core in the PNM device performs the sub-search based on ANNS in a subgraph including a subset of data vectors. For example, each PNM device can perform Figure 11, and in this case, “graph” in the following description may correspond to “subgraph” and “search” may correspond to “subsearch”.

[0111] In operation S1110, the method may include selecting a current vector. For example, a starting data vector may be selected as the current vector. The starting data vector may be a data vector in the graph selected randomly, an algorithmically selected vector, or a data vector at the center of the graph, but the present disclosure is not limited thereto. For example, Figure 1B , vector A is the starting data vector, and vector A is selected as the current vector. For example, when the candidate capacity is 2, the candidate can be initialized to include vector A (the candidate is not full at this time), and the distance L between the candidate data vector (for example, vector A) and the query vector in the candidate can be recorded. A .

[0112] In operation S1120, the method may include determining whether the current vector has been changed. If the current vector has not been changed compared to the previous search, the search process ends and the method proceeds to operation S1170. In operation S1170, the result is updated. For example, the search result is output. In a scenario where the PNM device performs a sub-search, the PNM device may return the sub-result to the host. If the current vector has been changed compared to the previous search, the method proceeds to operation S1130. According to another embodiment, the judgment condition in operation S1120 may also be, for example, "the candidate has not been updated." If the judgment is "yes" (for example, the candidate has not been updated), the method proceeds to operation S1170; otherwise, the method proceeds to operation S1130.

[0113] In operation S1130, the method may include performing a graph traversal. For example, the neighbor data vectors of the current vector may be traversed in the graph. Referring to FIG1 , the neighbor data vectors of vector A traversed in the graph are vectors B and C.

[0114] In operation S1140, the method may include performing a distance calculation. For example, calculating the distance between the neighbor data vector and the query vector. Figure 1B , calculate the distance L between vector B and vector C and the query vector respectively B and L C .

[0115] In operation S1150, the method may include performing candidate update. For example, among the neighbor data vectors and the existing candidate data vectors, select the candidate capacity (e.g., 2) data vectors closest to the query vector as new candidate data vectors, and update the candidate if there is a new candidate data vector different from the existing candidate data vector. A 、L B and L C1, the two data vectors closest to the query vector are vector B and vector C (L B <L C <L A ), update the candidate to include vector B and vector C, and record the distance L between the candidate data vector (vector B and vector C) and the query vector B and L C .

[0116] In operation S1160, the method may include selecting a candidate data vector closest to the query vector from the updated candidates as the current vector. After selecting the candidate data vector, the method returns to operation S1120. The candidate data vectors in the updated candidates include vector B and vector C, and vector B (L) closest to the query vector is selected. B <L C ) as the current vector, and returns to operation S1120.

[0117] After returning to operation S1120, since the current vector has been changed (from vector A to vector B), the process proceeds to operation S1130. The process continues to execute operations S1130 to S1160 until it is determined in operation S1120 that the current vector has not been changed.

[0118] Reference Figure 1B and Figure 11 , in the case where the candidate capacity is 2, the first round: in the example case where the current vector is vector A, the neighbor data vectors traversed in the graph are vector B and vector C, the candidate is updated to include vector B and vector C, and the current vector is switched to vector B. The second round: in the example case where the current vector is vector B, the neighbor data vector traversed in the graph is vector F (vector A has been traversed and can be no longer considered), the candidate is updated to include vector F and vector B (L F <L B <L C ), the current vector is switched to vector F; the third round: in the example case where the current vector is F, the neighbor data vector traversed in the graph is vector G (vector B has been traversed and can be no longer considered), and the candidate data vectors are updated to include vector G and vector F (L G <L F <L B ), the current vector switches to vector G; Round 4: In the example case where the current vector is vector G, the neighbor data vector traversed in the graph is vector E (vector F has been traversed and can be no longer considered). At this time, due to L G <L F <L E, the candidates are not updated and still include vector G and vector F, and the current vector is still vector G. At this time, the search end condition in operation S1120 is met, and operation S1170 is performed to return vector G and vector F in the candidates to the host.

[0119] After the above four rounds, the search is complete. Vector G and vector F are the two data vectors closest to the query vector in the figure. It should be understood that the above candidate capacity of 2 is only an example and the present disclosure is not limited thereto.

[0120] Return to reference Figure 10 In operation S1020, the method may include returning a sub-result based on the sub-search. For example, after the sub-search is completed, the PNM device returns the sub-result based on the sub-search to the host.

[0121] According to an embodiment of the present disclosure, returning a sub-result based on a sub-search to a host may include: returning a candidate data vector among the candidates to the host as the sub-result according to a return address of the sub-result.

[0122] According to an embodiment of the present disclosure, the return address of the sub-result may be an address in the host memory. After the sub-search is completed, the PNM device may return the sub-result to the host memory according to the return address of the sub-result. For example, the PNM device may return candidate data vectors (e.g., vector G and vector F) from the candidate to the host according to the return address of the sub-result (result).

[0123] As described above, in the retrieval method according to the embodiment of the present disclosure, first, a subgraph including a subset of data vectors is stored in the memory of the PNM device and the sub-retrieval is completed in the PNM device, thereby reducing the data transmission between the CPU and the memory during the retrieval; secondly, the sub-retrieval is offloaded to the PNM device, and the host only starts the sub-retrieval in the PNM device through the interface and performs the merging of a small number of sub-results, thereby reducing the load on the CPU; thirdly, the retrieval is divided into multiple sub-searches executed in parallel in the PNM device, and the reduction of graph density and candidate capacity reduces the computational amount of the retrieval and improves the retrieval performance.

[0124] Figure 12 FIG. 1 shows an overall data transmission flow chart of a retrieval method according to an embodiment of the present disclosure. Figure 12, in operation 12-1, the method includes receiving a query vector. For example, the host may receive a query vector input by a user. In operation 12-2, the method may include initiating a sub-search of each PNM device. For example, the host uses an interface (e.g., pnm_retrieval) to initiate a sub-search of each PNM device, and the retrieval agent module in the host may obtain interface-related parameters and initiate sub-searches of multiple PNM devices through the interface. In operation 12-3, the method may include executing a sub-search inside each PNM device. For example, each of the multiple PNM devices may execute a sub-search. For example, the retrieval core module in each PNM device may execute a sub-search on the query vector in a data subset stored in the PNM device memory (e.g., a subgraph including a data vector subset) based on the retrieval parameters passed by the interface. In operation 12-4, the method may include receiving sub-results from multiple PNM devices. For example, the host may receive sub-results. For example, the host may receive multiple sub-results based on the sub-search returned by multiple PNM devices. For example, the sub-results are stored in the host memory (e.g., Figure 5 In operation 12-5, the method may include merging the sub-results. For example, the host may merge the sub-results, and a merge result module in the host may merge the results. In operation 12-6, the method may include outputting the final result. For example, the host may output the merged search results.

[0125] Figure 13 A schematic diagram illustrating a host according to an embodiment of the present disclosure.

[0126] Reference Figure 13 The host 1300 may include a memory 1310 (e.g., a memory) and a processor 1320 (e.g., a CPU). For example, the memory 1320 may store instructions, and when the instructions are executed by the processor 1320, the processor 1320 may be prompted to perform the following operations: receive a query vector; control and initiate multiple near-memory processing (PNM) devices to perform sub-searches related to the query vector in their respective stored data subsets; receive multiple sub-results returned by the multiple PNM devices based on the sub-searches; and output a search result related to the query vector based on the multiple sub-results.

[0127] According to an embodiment of the present disclosure, when the instructions are executed by the processor 1320, the processor 1320 may be prompted to perform the following operations: providing search parameters to multiple PNM devices to enable the multiple PNM devices to perform sub-searches in parallel on the data subsets stored by each PNM device. For example, the search parameters include parameters related to the query vector and the return address of the sub-results.

[0128] According to an embodiment of the present disclosure, the plurality of PNM devices may be all or part of the PNM devices in a PNM device pool.

[0129] According to an embodiment of the present disclosure, when the instruction is executed by the processor 1320 , the processor 1320 may be prompted to perform the following operations: merging multiple sub-results to output a search result.

[0130] According to an embodiment of the present disclosure, a sub-search may be performed based on an approximate nearest neighbor search (ANNS). For example, the stored data subset may be a subgraph including a subset of data vectors. For example, the subgraph may be obtained by splitting a graph including all data vectors used for ANNS.

[0131] According to an embodiment of the present disclosure, the search parameters may further include a candidate capacity of the ANNS. For example, the candidate capacity may indicate the number of candidate data vectors in the candidate, and the candidate capacity may be less than a reference number of results. For example, when the instructions are executed by processor 1320, the processor 1320 may be prompted to perform the following operations: sort the multiple sub-results based on the distance between the multiple sub-results and the query vector, and output the reference number of sorted sub-results as the search result. For example, the reference number of results is a predetermined number of results.

[0132] Figure 14 A schematic diagram illustrating a PNM device according to an embodiment of the present disclosure is shown.

[0133] Reference Figure 14 PNM device 1400 includes a memory 1410 and a computing unit 1420 (e.g., PNM). For example, memory 1410 stores a data subset; computing unit 1420 can perform a sub-search on a query vector in the stored data subset; and after the sub-search is completed, returns a sub-result based on the sub-search to the host.

[0134] According to an embodiment of the present disclosure, the computing unit 1420 may perform a sub-search in the stored data subset according to search parameters for initiating the sub-search provided by the host. For example, the search parameters include: parameters related to the query vector and the return address of the sub-result.

[0135] According to an embodiment of the present disclosure, sub-searching can be performed based on approximate nearest neighbor search (ANNS), and the stored data subset can be a subgraph including a subset of data vectors. For example, the subgraph can be obtained by splitting the graph including all data vectors used for ANNS.

[0136] According to an embodiment of the present disclosure, the search parameters may further include the candidate capacity of the ANNS. For example, the candidate capacity may indicate the number of candidate data vectors in the candidate. For example, the calculation unit 1420 may search the subgraph for the candidate capacity data vectors closest to the query vector based on the parameters related to the query vector.

[0137] According to an embodiment of the present disclosure, the computing unit 1420 may select a starting data vector as the current vector in the subgraph, and initialize the candidates including the starting data vector; traverse the neighbor data vectors of the current vector in the subgraph; calculate the distance between the neighbor data vector and the query vector; update the candidates based on the comparison of the distances; select the candidate data vector closest to the query vector from the updated candidates as the current vector, and, when the current vector is changed, return to the step of traversing the neighbor data vectors of the current vector in the subgraph.

[0138] According to an embodiment of the present disclosure, the computing unit 1420 may select a candidate data vector closest to the query vector from the neighboring data vectors and the existing candidate data vectors as a new candidate data vector; and update the candidate if there is a new candidate data vector that is different from the existing candidate data vector.

[0139] According to an embodiment of the present disclosure, the calculation unit 1420 may return a candidate data vector from the candidates as a sub-result to the host according to the return address of the sub-result.

[0140] As described above, with the host and PNM device of the present disclosure, first, a subgraph including a subset of data vectors is stored in the memory of the PNM device and sub-search is completed in the PNM device, reducing data transmission between the CPU and the memory during the search; second, the sub-search is offloaded to the PNM device, and the host only starts the sub-search in the PNM device through the interface and performs the merging of a small number of sub-results, reducing the load on the CPU; third, the search is divided into multiple sub-searches executed in parallel in the PNM device, and the reduction of graph density and candidate capacity reduces the computational complexity of the search and improves the performance of the search.

[0141] Figure 15 Schematic diagram of a system 1000 to which a storage device is applied according to an embodiment.

[0142] Figure 15 The system 1000 may be, for example, a mobile system such as a portable communication terminal (e.g., a mobile phone), a smartphone, a tablet personal computer (PC), a wearable device, a healthcare device, or an Internet of Things (IoT) device. However, Figure 15 The system 1000 is not limited thereto, and may be, for example, a PC, a laptop computer, a server, a media player, or an automotive device (eg, a navigation device).

[0143] Reference Figure 15The system 1000 may include a main processor 1100, memories (e.g., 1200a and 1200b), and storage devices (e.g., 1300a and 1300b). In addition, the system 1000 may include at least one of an image capture device 1410, a user input device 1420, a sensor 1430, a communication device 1440, a display 1450, a speaker 1460, a power supply 1470, and a connection interface 1480.

[0144] The main processor 1100 may control all operations of the system 1000, including, for example, operations of other components included in the system 1000. The main processor 1100 may be implemented as, for example, a general-purpose processor, a dedicated processor, an application processor, or the like.

[0145] The main processor 1100 may include at least one central processing unit (CPU) core 1110 and a controller 1120 for controlling memories 1200a and 1200b and / or storage devices 1300a and 1300b. In some exemplary embodiments, the main processor 1100 may further include an accelerator 1130, which is a dedicated circuit for high-speed data operations such as, for example, artificial intelligence (AI) data operations. The accelerator 1130 may include, for example, a graphics processing unit (GPU), a neural processing unit (NPU), and / or a data processing unit (DPU), and may be implemented as a chip physically separated from other components of the main processor 1100.

[0146] Memories 1200a and 1200b may serve as main storage devices of system 1000. Although memories 1200a and 1200b may each include volatile memory, such as, for example, static random access memory (SRAM) and / or dynamic random access memory (DRAM), according to embodiments, memories 1200a and 1200b may each include non-volatile memory, such as, for example, flash memory, phase-change random access memory (PRAM), and / or resistive random access memory (RRAM). Memories 1200a and 1200b may be implemented in the same package as main processor 1100. Memory 1200b may also be a PNM device. For example, memory 1200a may be implemented in the same package as main processor 1100, while memory 1200b may be a PNM device external to the main processor, and there may be multiple PNM devices (e.g., PNM devices 1200b, 1200c, ...).

[0147] The memory devices 1300a and 1300b may function as non-volatile memory devices configured to store data regardless of whether power is supplied, and may have a larger storage capacity than the memories 1200a and 1200b. The memory devices 1300a and 1300b may include memory controllers (STRG CTRL) 1310a and 1310b, respectively, and non-volatile memories (NVMs) 1320a and 1320b, respectively, configured to store data under the control of the memory controllers 1310a and 1310b. Although the NVMs 1320a and 1320b may include V-NAND flash memories having a two-dimensional (2D) or three-dimensional (3D) structure, the NVMs 1320a and 1320b may include other types of NVMs, such as, for example, PRAM and / or RRAM.

[0148] The storage devices 1300a and 1300b may be physically separated from the main processor 1100 and included in the system 1000, or may be implemented in the same package as the main processor 1100. The storage devices 1300a and 1300b may be solid-state devices (SSDs) or memory cards and may be removably coupled to other components of the system 100 via an interface such as the connection interface 1480 described further below. The storage devices 1300a and 1300b may be devices to which a standard protocol such as Universal Flash Storage (UFS), embedded MultiMediaCard (eMMC), or NVMe is applied. However, the storage devices 1300a and 1300b are not limited thereto.

[0149] The image capturing device 1410 can capture still images or moving images and may include, for example, a camera, a camcorder, and / or a webcam.

[0150] The user input device 1420 may receive various types of data input by a user of the system 1000 and may include, for example, a touch pad, a keypad, a keyboard, a mouse, a microphone, and the like.

[0151] The sensor 1430 can detect various types of physical quantities that can be obtained from outside the system 1000 and convert the detected physical quantities into electrical signals. The sensor 1430 may include, for example, a temperature sensor, a pressure sensor, an illumination sensor, a position sensor, an acceleration sensor, a biosensor, and / or a gyroscope sensor.

[0152] The communication device 1440 may transmit and receive signals between other devices outside the system 1000 according to various communication protocols. The communication device 1440 may include, for example, an antenna, a transceiver, or a modem.

[0153] Display 1450 and speaker 1460 may serve as output devices configured to output visual information and auditory information, respectively, to a user of system 1000 .

[0154] The power supply device 1470 may appropriately convert power supplied from a battery embedded in the system 1000 and / or an external power source, and supply the converted power to each component of the system 1000 .

[0155] The connection interface 1480 may provide a connection between the system 1000 and an external device that is connected to the system 1000 and is capable of transmitting and receiving data to and from the system 1000. The connection interface 1480 may be implemented by using various interface schemes, such as, for example, Advanced Technology Attachment (ATA), Serial ATA (SATA), External Serial ATA (e-SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), Peripheral Component Interconnect (PCI), PCI express (PCIe), NVMe, IEEE 1394, Universal Serial Bus (USB) interface, Secure Digital (SD) card interface, MultiMediaCard (MMC) interface, Embedded MultiMediaCard (eMMC) interface, UFS interface, Embedded UFS (eUFS) interface, and Compact Flash (CF) card interface, etc.

[0156] According to an embodiment of the present disclosure, a system (e.g., 1000) employing a near memory processing (PNM) device is provided, comprising: a main processor (e.g., 1100); a memory (e.g., 1200a); a plurality of PNM devices (e.g., 1200b, 1200c, ...); and storage devices (e.g., 1300a and 1300b). For example, the PNM devices are configured to perform the retrieval method described above.

[0157] Figure 16 is a block diagram of a host storage system 10 according to an embodiment.

[0158] The host storage system 10 may include a host 100 and a storage device 200. The storage device 200 may include a memory controller 210 and an NVM 220. According to an embodiment, the host 100 may include a host controller 110 and a host memory 120. The host memory 120 may be used as a buffer memory configured to temporarily store data to be transmitted to or received from the storage device 200.

[0159] The storage device 200 may include a storage medium configured to store data in response to a request from the host 100. As an example, the storage device 200 may include at least one of an SSD, an embedded memory, and a removable external memory. In the example case where the storage device 200 is an SSD, the storage device 200 may be a device that complies with the NVMe standard. In the example case where the storage device 200 is an embedded memory or an external memory, the storage device 200 may be a device that complies with the UFS standard or the eMMC standard. Both the host 100 and the storage device 200 may generate a packet according to the adopted standard protocol and may send the packet.

[0160] In the example case where the NVM 220 of the storage device 200 includes flash memory, the flash memory may include a 2D NAND memory array or a 3D (or vertical) NAND (VNAND) memory array. As another example, the storage device 200 may include various other types of NVM. For example, the storage device 200 may include magnetic random access memory (MRAM), spin transfer torque MRAM, conductive bridge RAM (CBRAM), ferroelectric RAM (FRAM), PRAM, RRAM, and various other types of memory.

[0161] According to an exemplary embodiment, the host controller 110 and the host memory 120 may be implemented as separate semiconductor chips. However, the present disclosure is not limited thereto, and therefore, according to another embodiment, the host controller 110 and the host memory 120 may be integrated into the same semiconductor chip. As an example, the host controller 110 may be any one of a plurality of devices included in an application processor (AP). The AP may be implemented as, for example, a system on a chip (SoC). In addition, the host memory 120 may be an embedded memory included in the AP or a memory device external to the AP. The host may also include a host memory 120 implemented outside the host, for example, the host memory 120 may be a plurality of near memory processing (PNM) devices (e.g., PNM devices 120a, 120b, ...) outside the host.

[0162] The host controller 110 may manage an operation of storing data (eg, write data) of the buffer area of ​​the host memory 120 in the NVM 220 or an operation of storing data (eg, read data) of the NVM 220 in the buffer area.

[0163] The memory controller 210 may include a host interface 211, a memory interface 212, a CPU 213, a flash translation layer (FTL), a packet manager 215, a buffer memory 216, an error correction code (ECC) engine 217, and an advanced encryption standard (AES) engine 218. The memory controller 210 may further include a working memory in which the FTL 214 is loaded. The CPU 213 may execute the FTL 214 to control data write and read operations on the NVM 220.

[0164] The host interface 211 can transmit and receive packets to and from the host 100. The packets transmitted from the host 100 to the host interface 211 may include commands or data to be written to the NVM 220, etc. The packets transmitted from the host interface 211 to the host 100 may include responses to commands or data read from the NVM 220, etc. The memory interface 212 can transmit data to be written to the NVM 220 to the NVM 220 or receive data read from the NVM 220. The memory interface 212 may be configured to comply with a standard protocol such as, for example, Toggle or Open NAND Flash Interface (ONFI).

[0165] The FTL 214 can perform various functions, such as, for example, address mapping, wear leveling, and garbage collection. The address mapping operation may be an operation that converts a logical address received from the host 100 into a physical address for actually storing data in the NVM 220. The wear leveling operation may be a technique for preventing or reducing excessive degradation of a particular block by ensuring uniform use of the blocks of the NVM 220. For example, the wear leveling operation may be implemented using firmware techniques that balance the erase counts of physical blocks. The garbage collection operation may be a technique for ensuring available capacity in the NVM 220 by erasing existing blocks after copying valid data from the existing blocks to new blocks.

[0166] The packet manager 215 may generate a packet according to a protocol agreed with an interface of the host 100, or parse various types of information from a packet received from the host 100. In addition, the buffer memory 216 may temporarily store data to be written to or read from the NVM 220. Although the buffer memory 216 may be a component included in the memory controller 210, the present disclosure is not limited thereto, and thus, according to some embodiments, the buffer memory 216 may be provided outside the memory controller 210.

[0167] The ECC engine 217 may perform error detection and correction operations on read data read from the NVM 220. For example, the ECC engine 217 may generate parity bits for write data to be written to the NVM 220, and the generated parity bits may be stored together with the write data in the NVM 220. During reading of data from the NVM 220, the ECC engine 217 may correct errors in the read data by using the read data and the parity bits read from the NVM 220, and output the error-corrected read data.

[0168] The AES engine 218 may perform at least one of an encryption operation and a decryption operation on data input to the memory controller 210 by using a symmetric key algorithm.

[0169] According to an embodiment of the present disclosure, a host storage system (e.g., 10) is provided, comprising: a host (e.g., 100); and a plurality of near memory processing (PNM) devices. For example, the host may execute the retrieval method applied to the host as described above, and the PNM devices may execute the retrieval method applied to the PNM devices as described above.

[0170] Figure 17 is a diagram of a data center 3000 to which a storage device is applied, according to an exemplary embodiment.

[0171] Platform - Server (Application / Storage)

[0172] Reference Figure 17 , the data center 3000 can be a facility that collects various types of data and provides services, and can be called a data storage center. The data center 3000 can be a system for operating a search engine and a database, and can be a computing system used by a company (such as a bank) or a government agency. The data center 3000 may include application servers 3100 to 3100n and storage servers 3200 to 3200m, where n and m are positive integers. According to an exemplary embodiment, the number of applications 3100 to 3100n and the number of storage servers 3200 to 3200m can be selected differently. The number of application servers 3100 to 3100n and the number of storage servers 3200 to 3200m can be different from each other.

[0173] The application server 3100 or the storage server 3200 may include processors 3110 and 3210, at least one of memories 3120 and 3220, at least one of switches 3130 to 3130n, at least one of network interface cards (NICs) 3140 to 3140n and 3240 to 3240m, at least one of DRAMs 3253 to 3253m, and at least one of controllers 3251 to 3251m. The memory 3120 may also include multiple near memory processing (PNM) devices (e.g., 3120a, 3120b, ...). The storage server 3200 will now be described as an example. The processor 3210 may control all operations of the storage server 3200, access the memory 3220, and execute instructions and / or data loaded into the memory 3220. The memory 3220 may be, for example, double data rate synchronous DRAM (DDR SDRAM), high bandwidth memory (HBM), a hybrid memory cube (HMC), a dual in-line memory module (DIMM), an Optane DIMM, or a non-volatile DIMM (NVM DIMM). In some exemplary embodiments, the number of processors 3210 and memory 3220 included in the storage server 3200 may be selected differently. In one exemplary embodiment, the processors 3210 and memory 3220 may provide a processor-memory pair. In one exemplary embodiment, the number of processors 3210 and the number of memories 3220 may differ from each other. The processor 3210 may include a single-core processor or a multi-core processor. The above description of the storage server 3200 may similarly apply to the application server 3100. In some embodiments, the application server 3100 may not include a storage device 3150. The storage server 3200 may include at least one storage device 3250. Depending on the embodiment, the number of storage devices 3250 included in the storage server 3200 may be selected differently.

[0174] Platform part - Network

[0175] The application servers 3100 to 3100n can communicate with the storage servers 3200 to 3200m via a network 3300. The network 3300 can be implemented using Fibre Channel (FC) or Ethernet. In this case, FC can be a medium for relatively high-speed data transmission, and optical switches with high performance and high availability can be used. Depending on the access method of the network 3300, the storage servers 3200 to 3200m can be configured as file storage, block storage, or object storage.

[0176] In one embodiment, network 3300 may be a network dedicated to storage, such as a storage area network (SAN). For example, the SAN may be an FC-SAN, which uses an FC network and is implemented according to the FC protocol (FCP). As another example, the SAN may be an Internet Protocol (IP)-SAN, which uses a Transmission Control Protocol (TCP) / IP network and is implemented according to the SCSI over TCP / IP or Internet SCSI (iSCSI) protocol. In one embodiment, network 3300 may be a general-purpose network, such as a TCP / IP network. For example, network 3300 may be implemented according to protocols such as FC over Ethernet (FCoE), Network Attached Storage (NAS), and NVMe over Fabric (NVMe-oF).

[0177] Hereinafter, description will be mainly made of the application server 3100 and the storage server 3200. The description of the application server 3100 may be applied to another application server 3100n, and the description of the storage server 3200 may be applied to another storage server 3200m.

[0178] The application server 3100 can store data requested to be stored by a user or client in one of the storage servers 3200 to 3200m via the network 3300. In addition, the application server 3100 can obtain data requested to be read by a user or client from one of the storage servers 3200 to 3200m via the network 3300. For example, the application server 3100 can be implemented as a network server or a database management system (DBMS).

[0179] The application server 3100 can access the memory 3120n or storage device 3150n included in another application server 3100n through the network 3300. However, the present disclosure is not limited thereto, and according to an embodiment, the application server 3100 can access the memory 3220 to 3220m or storage devices 3250 to 3250m included in the storage servers 3200 to 3200m through the network 3300. Therefore, the application server 3100 can perform various operations on the data stored in the application servers 3100 to 3100n and / or the storage servers 3200 to 3200m. For example, the application server 3100 can execute instructions for moving or copying data between the application servers 3100 to 3100n and / or the storage servers 3200 to 3200m. In this case, data may be moved from the storage devices 3250 to 3250m of the storage servers 3200 to 3200m through the storage 3220 to 3220m of the storage servers 3200 to 3200m or directly to the storage 3120 to 3120n of the application servers 3100 to 3100n. The data moved over the network 3300 may be encrypted data for security or privacy.

[0180] Organic Relationship-Interface Structure / Type

[0181] The storage server 3200 will now be described as an example. The interface 3254 can provide a physical connection between the processor 3210 and the controller 3251 and a physical connection between the network interface card (NIC) 3240 and the controller 3251. For example, the interface 3254 can be implemented using a direct attached storage (DAS) solution, wherein the storage device 3250 is directly connected to a dedicated cable. For example, the interface 3254 can be implemented using various interface solutions, such as ATA, SATA, e-SATA, SCSI, SAS, PCI, PCIe, NVMe, IEEE 1394, USB interface, SD card interface, MMC interface, eMMC interface, UFS interface, eUFS interface, and CF card interface.

[0182] The storage server 3200 may further include a switch 3230 and a network interconnect (NIC) 3240. The switch 3230 may selectively connect the processor 3210 to the storage device 3250 or selectively connect the NIC 3240 to the storage device 3250 under the control of the processor 3210.

[0183] In one embodiment, the NIC 3240 may include a network interface card and a network adapter. The NIC 3240 may be connected to the network 3300 via, for example, a wired interface, a wireless interface, a Bluetooth interface, or an optical interface. The NIC 3240 may include internal memory, a digital signal processor (DSP), and a host bus interface, and may be connected to the processor 3210 and / or the switch 3230 via the host bus interface. The host bus interface may be implemented as one of the above-described examples of the interface 3254. In one embodiment, the NIC 3240 may be integrated with at least one of the processor 3210, the switch 3230, and the storage device 3250.

[0184] Organic Relationship-Interface Operation

[0185] In the storage servers 3200 to 3200m or the application servers 3100 to 3100n, the processor may send a command to the storage devices 3150 to 3150n and 3250 to 3250m or the memories 3120 to 3120n and 3220 to 3220m and program or read data. In this case, the data may be data on which errors have been corrected by an ECC engine. The data may be data on which a data bus inversion (DBI) operation or a data masking (DM) operation has been performed and may include cyclic redundancy code (CRC) information. The data may be data encrypted for security or privacy.

[0186] Memory devices 3150 to 3150n and 3250 to 3250m can send control signals and command / address signals to NAND flash memory devices 3252 to 3252m in response to a read command received from a processor. Therefore, in the example case of reading data from NAND flash memory devices 3252 to 3252m, a read enable (RE) signal can be input as a data output control signal, thereby outputting the data to the DQ bus. The RE signal can be used to generate a data strobe signal DQS. Depending on the rising or falling edge of the write enable (WE) signal, the command and address signals can be latched in the page buffer.

[0187] Product Section - Basic Operations of SSD

[0188] The controller 3251 can control all operations of the storage device 3250. In one embodiment, the controller 3251 may include an SRAM. The controller 3251 can write data to the NAND flash memory device 3252 in response to a write command, or read data from the NAND flash memory device 3252 in response to a read command. For example, the write command and / or read command can be provided by the processor 3210 of the storage server 3200, the processor 3210m of another storage server 3200m, or the processors 3110 and 3110n of the application servers 3100 and 3100n. The DRAM 3253 can temporarily store (or buffer) data to be written to or read from the NAND flash memory device 3252. The DRAM 3253 can also store metadata. Here, the metadata can be user data or data generated by the controller 3251 for managing the NAND flash memory device 3252. The storage device 3250 may include a secure element (SE) for security or privacy.

[0189] According to an exemplary embodiment of the present disclosure, a data center system (e.g., 3000) is provided, comprising: a plurality of application servers (3100 to 3100n); and a plurality of storage servers (e.g., 3200 to 3200m). For example, each application server includes a plurality of near memory processing (PNM) devices. For example, the PNM devices are configured to execute the retrieval method described above.

[0190] As is conventional in the field of the present disclosure, embodiments are described and shown in the accompanying drawings according to functional blocks, units and / or modules. It will be understood by those skilled in the art that these blocks, units and / or modules are physically implemented by electronic (or optical) circuits (such as logic circuits, discrete components, microprocessors, hard-wired circuits, memory elements, wiring connections, etc.), which can be formed using semiconductor-based manufacturing techniques or other manufacturing techniques. In the case where blocks, units and / or modules are implemented by microprocessors or the like, they can be programmed using software (e.g., microcode) to perform the various functions discussed herein, and they can optionally be driven by firmware and / or software. However, the present disclosure is not limited thereto, and according to an embodiment, each block, unit and / or module can be implemented by dedicated hardware, or implemented as a combination of dedicated hardware that performs some functions and a processor (e.g., one or more programmed microprocessors and associated circuits) that performs other functions.

[0191] According to an embodiment of the present disclosure, a computer-readable storage medium storing a computer program is provided. For example, when the computer program is executed by a processor, the retrieval method described above is implemented.

[0192] According to an embodiment of the present disclosure, an electronic device is provided, including: a processor and a memory, in which a computer program is stored. For example, when the computer program is executed by the processor, the retrieval method described above is implemented.

[0193] According to an embodiment of the present disclosure, a computer-readable storage medium storing a computer program may also be provided. When the program is run, the retrieval method according to an embodiment of the present disclosure may be executed. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, for example, a multimedia card, a secure digital (SD) card and / or an ultra-fast digital (XD) card), a magnetic tape, a floppy disk, a magneto-optical data storage device, an optical data storage device, a hard disk, a solid state disk and / or any other device, any other device configured to store a computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and / or data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In one example, the computer program and any associated data, data files and / or data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0194] According to the retrieval method, host and PNM device of the exemplary embodiments of the present disclosure, first, a subgraph including a subset of data vectors is stored in the memory of the PNM device and the sub-search is completed in the PNM device, thereby reducing data transmission between the CPU and the memory during the retrieval; second, the sub-search is offloaded to the PNM device, and the host only starts the sub-search in the PNM device through the interface and performs the merging of a small number of sub-results, thereby reducing the load on the CPU; third, the search is divided into multiple sub-searches executed in parallel in the PNM device, and the reduction of graph density and the reduction of candidate capacity reduce the computational amount of the search and improve the performance of the search.

[0195] While the present disclosure has been particularly shown and described with reference to embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the following claims.

Claims

1. A retrieval method, performed by a host device, comprising: Receive a query vector; Starting a plurality of near memory processing (PNM) devices to perform a sub-search corresponding to the query vector; receiving a plurality of sub-results, each of the plurality of sub-results being provided by one of a plurality of PNM devices based on the sub-search performed on a subset of data stored in the respective PNM device; as well as Based on the multiple sub-results, a search result corresponding to the query vector is output.

2. The method according to claim 1, wherein Starting the plurality of PNM devices to perform a sub-search corresponding to the query vector includes: Providing search parameters to the plurality of PNM devices to enable the plurality of PNM devices to perform the sub-searches in parallel in the data subsets stored in the respective PNM devices, wherein the search parameters include parameters related to the query vector and return addresses of sub-results.

3. The method according to claim 1, wherein The multiple PNM devices are all or part of the PNM devices in a PNM device pool.

4. The method according to claim 1, wherein Outputting the search results includes: The multiple sub-results are merged to output the search result.

5. The method according to claim 2, wherein: The sub-search is performed based on the Approximate Nearest Neighbor Search (ANNS) method. The data subset stored in each PNM device is a subgraph including a subset of data vectors. The subgraphs are obtained by splitting a graph including all data vectors used in the ANNS method.

6. The method according to claim 5, wherein: The search parameters also include the candidate capacity of the ANNS method, wherein the candidate capacity indicates the number of candidate data vectors in the candidate list and the candidate capacity is less than a reference number of results, Outputting the search results includes: sorting the sub-results based on distances between the sub-results and the query vector, and The reference number of sorted sub-results is output as the search result.

7. A retrieval method, performed by a near memory processing (PNM) device, comprising: receiving a request to perform a sub-search corresponding to a query vector; performing said sub-search on a subset of data stored in the PNM device; Sub-results based on the sub-search are sent to a host device.

8. The method of claim 7, wherein: Executing the sub-search includes: performing the sub-search in the data subset according to search parameters provided by the host device to initiate the sub-search, The search parameters include: parameters related to the query vector and the return address of the sub-result.

9. The method of claim 8, wherein: The sub-search is performed based on the Approximate Nearest Neighbor Search (ANNS) method, and the data subset is a subgraph including a subset of data vectors, and The subgraphs are obtained by splitting a graph including all data vectors used in the ANNS method.

10. The method of claim 9, wherein: The search parameters also include a candidate capacity of the ANNS method, wherein the candidate capacity indicates the number of candidate data vectors in the candidate list, and Wherein, performing the sub-search in the data subset according to the search parameter includes: According to the search parameter, a first number of data vectors closest to the query vector are searched in the subgraph, wherein the first number of data vectors satisfies the candidate capacity.

11. The method according to claim 10, wherein: Searching the subgraph for a first number of data vectors closest to the query vector includes: selecting a starting data vector in the subgraph as a current vector, and initializing a candidate list to include the starting data vector; Traversing neighbor data vectors of the current vector in the subgraph; Calculating the distance between the neighbor data vector and the query vector; updating the candidate list based on the comparison of the distances; Select the candidate data vector closest to the query vector in the updated candidate list as the current vector, and Based on the change of the current vector, the operation of traversing the neighboring data vectors of the current vector in the subgraph is repeated.

12. The method of claim 11, wherein: Based on the comparison of the distances, the candidate list is updated, including: Selecting a first number of data vectors closest to the query vector from among neighbor data vectors and existing candidate data vectors as new candidate data vectors; and Based on the new candidate data vectors including new candidate data vectors that are different from the existing candidate data vectors, the candidate list is updated.

13. The method of claim 10, wherein: Sending a sub-result based on the sub-search to a host device, comprising: According to the return address of the sub-result, the candidate data vectors in the candidate list are sent to the host device as the sub-result.

14. A host device comprising: a memory storing one or more instructions; as well as A processor configured to execute the one or more instructions to perform the following operations: Receive a query vector; Starting a plurality of near memory processing (PNM) devices to perform a sub-search corresponding to the query vector; receiving a plurality of sub-results, each of the plurality of sub-results being provided by one of a plurality of PNM devices based on the sub-search performed on a subset of data stored in the respective PNM device; as well as Based on the multiple sub-results, a search result corresponding to the query vector is output.

15. A near memory processing (PNM) device, comprising: Memory, which stores a subset of data; as well as The processor is configured to: receiving a request to perform a sub-search corresponding to a query vector; performing said sub-search on a subset of data stored in the PNM device; and Sub-results based on the sub-search are sent to a host device.

16. A system comprising: Main processor; Memory; Multiple near memory processing (PNM) devices; as well as storage device; Each of the plurality of PNM devices is configured to execute the retrieval method according to any one of claims 7 to 13.

17. A storage system comprising: host device; as well as Multiple Near Memory Processing (PNM) devices, Wherein, the host device is configured as follows: Receive a query vector; Initiate multiple PNM devices to perform sub-searches corresponding to the query vector; receiving a plurality of sub-results, each of the plurality of sub-results being provided by one of the plurality of PNM devices based on the sub-search; Based on the multiple sub-results, outputting a search result corresponding to the query vector; and Each of the plurality of PNM devices is configured as: performing said sub-search on a subset of data stored in a respective PNM device; The sub-results based on the sub-search are transmitted to the host device.

18. A data center system, comprising: Multiple application servers; as well as Multiple storage servers, Each application server includes a plurality of near memory processing (PNM) devices, wherein each of the plurality of PNM devices is configured to execute the retrieval method according to any one of claims 7 to 13.

19. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, the search method according to any one of claims 1 to 13 is implemented.

Images