Parallel pruning and batch sorting for similarity search accelerators

Abstract

To provide a computing system using a similarity processing engine for performing parallel pruning and similarity calculation for a candidate vector and a query vector, a program, a semiconductor device, and storage media.SOLUTION: A method of similarity retrieval processing in a similarity retrieval architecture having a plurality of processing engines uses a first processing engine of a plurality of processing engines to determine a first partial similarity measurement value based on a first portion of a query vector and a first portion of a first candidate vector, uses a second processing engine of the plurality of processing engines to determine a total similarity measurement value based on the query vector and a second candidate vector, and uses the first processing engine to determine whether to compare a second portion of the query vector with a second portion of the first candidate vector based on the first partial similarity measurement value and the total similarity measurement value.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] Embodiments generally relate to a processing architecture that performs parallel pruning and similarity calculations for candidate vectors and query vectors using similarity processing engines, including shared heap hardware that sorts results from the parallel similarity processing engines. [Background technology]

[0002] Content-based similarity search (e.g., similarity search) can be performed by machine learning (ML) and / or artificial intelligence (AI) applications (e.g., recommendation engines, visual search engines, drug discovery, etc.). For example, a database may contain a large number (e.g., billions) of high-dimensional candidate vectors. A query vector q of the same dimension, format, and size (e.g., 512 bytes) can be matched against the database (e.g., based on some similarity function such as a Euclidean similarity measure) to identify database vectors that are similar and / or closest to the query vector q. For example, a content-based image retrieval (CBIR) system may identify similar images in a database using a query image that is decomposed into a query vector and then matched against candidate vectors representing similar images. The feature extraction stage may involve deep learning models. Moreover, in modern applications, these vectors may represent broad categories such as the content of images, text, web searches, protein sequences, faces, sounds, or bioinformatics data, which are extracted and summarized by deep learning systems. [Brief explanation of the drawings]

[0003] Various advantages of the embodiments will become apparent to those skilled in the art upon reading the following specification and appended claims, and upon review of the following drawings.

[0004] [Figure 1] FIG. 1 is a diagram of an example near-memory similarity searching architecture according to one embodiment.

[0005] [Figure 2A] FIG. 1 is a diagram of an example architecture for far-memory similarity search and matching according to one embodiment. [Figure 2B] FIG. 1 is a diagram of an example architecture for far-memory similarity search and matching according to one embodiment.

[0006] [Figure 3] 1 is a flowchart of an example method of similarity search processing with pruning, according to one embodiment.

[0007] [Figure 4] FIG. 10 is a timing diagram of an example of a similarity search using a processing engine according to one embodiment.

[0008] [Figure 5] 1 is a flowchart of an example method for query streaming according to one embodiment.

[0009] [Figure 6] 1 is a flowchart of an example of a similarity calculation method according to an embodiment.

[0010] [Figure 7] FIG. 2 is a diagram of an example heap memory structure according to one embodiment.

[0011] [Figure 8] 1 is a flowchart of an example method for inserting a new node entry into an unfilled heap memory structure according to one embodiment.

[0012] [Figure 9] 1 is a flowchart of an example method for inserting a new node entry into a filled heap memory structure according to one embodiment.

[0013] [Figure 10]FIG. 1 is a block diagram of an example of a similarity search improvement computing system according to one embodiment.

[0014] [Figure 11] 1 illustrates an example of a semiconductor device according to an embodiment.

[0015] [Figure 12] FIG. 2 is a block diagram of an example of a processor according to an embodiment.

[0016] [Figure 13] FIG. 1 is a block diagram of an example of a multiprocessor-based computing system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Referring now to FIG. 1 , a similarity search architecture 100 (e.g., a system-on-chip) implements an improved similarity and early pruning search process that runs with reduced latency, lower bandwidth, and reduced computational resources. In particular, the similarity search architecture 100 performs parallel similarity calculations using an array of similarity processing engines (PEs) 110 (e.g., configurable logic, fixed-function logic hardware, processing elements, execution units, etc.) to identify closest similarity matches between first through Nth query vectors 106 a through 106 n and candidate vectors V000 through Vn03. As will be described in further detail, the similarity PEs 110 perform early pruning (e.g., bypassing) to discard similarity processing at an early stage and reduce computational resources and processing power. Furthermore, the similarity PEs 110 operate using a near-memory architecture including a memory area 108 to reduce bandwidth and communication latency. Moreover, the architecture 100 may operate over batches of query vectors 106 while the candidate vectors V000-Vn03 are stored in memory area 108 to avoid high latency data retrieval of the candidate vectors V000-Vn03 from long-term storage.

[0018] In particular, the similarity PE 110 may determine various similarity measures (e.g., Manhattan distance, Euclidean distance, Minkowski and Hamming distance, cosine and / or dot product similarity). Similarity may be determined by distance metrics such as Manhattan distance, Euclidean distance, Minkowski and Hamming distance, such that lower distances correspond to higher similarities. In some examples, the PE 110 may prune (e.g., bypass) future similarity calculations (e.g., similarity measures) between each candidate vector among the candidate vectors V000-Vn03 and each query vector among the query vectors 106 if the partially calculated distance therebetween is higher than a threshold (e.g., the longest distance, as determined from the top-k distances, and / or the lowest similarity score). Such pruning occurs at an early stage before all similarity measures between the vector features of each candidate vector and the vector features of each query vector are determined, and a decision is made as to whether to ignore each candidate vector. By doing so, computational resources and latency may be reduced without reducing accuracy. While each of memory areas 108 includes four candidate vectors V000-Vn03, it will be understood that such number is exemplary and that embodiments as described herein are not limited thereto. In fact, each of memory areas 108 may store any number of candidate vectors (e.g., M number) and operate in a manner similar to that described herein.

[0019] Additionally, some embodiments include first through Nth memory areas 108a-108n (e.g., static random access memory banks), each storing a subset of the candidate vectors V000-Vn03 and each dedicated to one of the similarity PEs 110. The multiple similarity PEs 110 may efficiently execute in parallel and based on different ones of the candidate vectors V000-Vn03 retrieved from the first through Nth memory areas 108a-108n. By doing so, latency may be further reduced because each of the first through Nth similarity PEs 110a-110n may be less likely to idle and wait due to blocking and waiting for the calculations of the other similarity PEs 110a-110n. For example, each of the first to N-th similarity PEs 110a to 110n may operate independently of the other first to N-th similarity PEs 110a to 110n, and perform a similarity search on a subset of candidate vectors V000 to Vn03.

[0020] Moreover, the first through Nth similarity PEs 110a through 110n may serially process batches of the first through Nth query vectors 106a through 106n. By doing so, memory fetches and power consumption may be reduced because the candidate vectors V000 through Vn03 may remain in the first through Nth memory areas 108a through 108n throughout the processing of the first through Nth query vectors 106a through 106n. For example, the similarity PE 110 may access a continuous query stream of query vectors 106 to perform a similarity search.

[0021] As described herein, a feature may be a useful piece of information extracted from data. The size of a feature is measured in bytes (e.g., 1 byte) and may also be referred to as a dimension. A high-dimensional input, such as an image, may be reduced to a reduced number of dimensions or features. A vector (e.g., a query vector and / or a candidate vector) includes multiple features to form a feature vector. The size of the feature vector is determined by the number of features and the size of each feature (e.g., 1 byte, INT8 format size, INT16 format size, INT32 format size, FP32 format size, BF16 format size). A query (or query vector) may be a feature vector extracted from a running application or a user search. For example, if a face detection application is used, the query vector may be a feature vector extracted from a query face, and the query vector may be compared against candidate vectors (each representing a candidate face) to identify matching faces.

[0022] In an embodiment, the similarity PE 110 determines a distance representing the similarity between two vectors (e.g., quantifies the similarity between the two vectors). That is, the similarity PE 110 calculates the distance or similarity between each query vector of the query vectors 106 and each candidate vector of the candidate vectors V000-Vn03 at a given time. For example, a single similarity PE of the multiple similarity PEs 110 may run for a certain number (e.g., 512) clock cycles to compare each query vector to each candidate vector, assuming a corresponding feature size (e.g., 512 bytes if the data path is 1 byte wide).

[0023] As described, the similarity PE 110 determines distance as a similarity measure. It will be understood that embodiments as described herein may determine other similarity measures and operate similarly to those described herein for distance. In this manner, the architecture 100 exploits the mathematical properties of distance and similarity algorithms to recursively calculate similarity across features in the query vector 106 and candidate vectors V000-Vn03.

[0024] Additionally, embodiments implement a heap hardware engine 114 that efficiently sorts results from the parallel first through Nth similarity PEs 110a through 110n. The heap hardware engine 114 enables batch sorting using a device for a hardware-friendly implementation of the heap algorithm to handle multiple parallel queries and entries.

[0025] As shown, the query buffer 102 includes first through Nth queries 106a through 106n. The query buffer 102 includes a scheduler 104 (e.g., a finite state machine) that schedules the queries, consisting of the first through Nth query vectors 106a through 106n, to be streamed to the similarity PE 110. The query buffer 102 stores a set of query vectors 106 (e.g., queries). The entire candidate vector database can be compared against the query vector 106 in sequence.

[0026] The similarity search processor 116 (e.g., a dedicated processor and / or accelerator architecture) receives one query vector from the first through Nth query vectors 106a through 106n at a time and performs the similarity search process. After completing processing for one query vector, the similarity search processor 116 receives another query from the first through Nth query vectors 106a through 106n and performs the similarity search process. As described below, the similarity search processor 116 stores a vector database and matches the vector database against the first through Nth query vectors 106a through 106n. The vector database may be a large collection of feature vectors (e.g., a face database includes feature vectors extracted from each face in a large population). The feature vectors in the vector database are candidate vectors V000 through Vn03. The similarity search processor 116 may match the first through Nth query vectors 106a through 106n (e.g., query faces) against the candidate vectors V000 through Vn03 (e.g., faces) to identify the most similar matches (e.g., identify matches between faces).

[0027] The similarity search processor 116 includes a memory area 108 (e.g., a memory bank). In this example, the similarity search processor 116 may perform near-memory calculations using multiple similarity PEs 110 operating in parallel. For example, a vector database including candidate vectors V000 to Vn03 may be stored in the storage area 108. The candidate vectors V000 to Vn03 may be of a size that can be accommodated within the memory area 108, thereby enabling all of the candidate vectors V000 to Vn03 to be efficiently contained in the memory area 108 (e.g., on-board memory). Therefore, each of the first to Nth memory areas 108a to 108n stores "a" number of vectors from the vector database. Therefore, the first to Nth similarity PEs 110a to 110n access "a*N" number of candidate vectors V000 to Vn03, where N is the total number of the first to Nth memory areas 108a to 108n.

[0028] As shown, each of the first through Nth memory areas 108a-108n is connected to one of the first through Nth similarity PEs 110a-110n. For example, the first memory area 108a is connected to and dedicated to the first similarity PE 110a, the second memory area 108b is connected to and dedicated to the second similarity PE 110b, and so on; the Nth memory area 108n is connected to and dedicated to the Nth similarity PE 110n. For example, the first similarity PE 110a is blocked and / or prohibited from accessing the second through Nth memory areas 108b-108n that are dedicated to other PEs among the second through Nth similarity PEs 110b-110n. Therefore, each of the first through Nth similarity PEs 110a through 110n has access to and operates on only a subset of the candidate vectors V000 through Vn03.

[0029] The scheduler 104 streams the queries 106 to the similarity PEs 110 in a daisy-chain manner. For example, the scheduler 104 may stream a first query vector 106a to a first similarity PE 110a (which may occur over several clock cycles). The first similarity PE 110a may receive the first query vector 106a and then provide the first query vector 106a to a second similarity PE 110b over one or more clock cycles, and so on until the Nth similarity PE 110n receives the first query vector 106a from the preceding one of the similarity PEs 110.

[0030] The first similarity PE 110a may initiate a similarity search when the first query vector 106a is received. For example, the first similarity PE 110a may compare the first query vector 106a with the candidate vector V000 to determine the similarity between the first query vector 106a and the candidate vector V000. As described, the similarity may be determined by a distance metric such as Manhattan distance, Euclidean distance, Minkowski distance, or Hamming distance, such that the lower the distance, the higher the similarity. Each of the first through Nth similarity PEs 110a through 110n may perform a similar calculation of the similarity when the first query vector 106a is received. For example, the second similarity PE 110b may generate the distance between the first query vector 106a and the candidate vector V010, the Nth similarity PE 110n may generate the distance between the first query vector 106a and the candidate vector Vn00, and so on.

[0031] Once the total distance is calculated, if it is less than the maximum distance (described further below), it may be sent to the hardware heap engine 114 and through the result engines 112, which are daisy-chained together. For example, after the first similarity PE 110a calculates the total distance between vector V000 and the first query vector 106a, the first similarity PE 110a associates the total distance with the vector ID of vector V000 and sends it to the first result engine 112a. Sending the vector ID (rather than the entire vector V000) reduces bandwidth and makes it easier to identify at a later point in time. The first result engine 112a sends the vector ID and total distance to the second result engine 112b, which in turn sends the vector ID and total distance to the subsequent one of the result engines 112, until the Nth result engine 112n is reached. The Nth result engine 112n sends the vector ID and total distance to a buffer 114a (e.g., an elastic buffer), which may temporarily store the vector ID and total distance until the heap controller 114b is available. The heap controller 114b may receive the vector ID and total distance when the heap controller 114b is available. The heap controller 114b stores the vector ID and total distance in one of nodes 0-n (any number of nodes may be used) of the heap memory 114c.

[0032] In some examples, nodes 0 to n may be arranged as a tree data structure. For example, the heap memory 114c may store nodes 0 to n as a binary tree data structure that holds the largest (e.g., a max-heap binary tree) / smallest element (e.g., a min-heap binary tree) in the root of the tree. The configuration of the binary data tree structure may be set as the largest or smallest based on a distance or similarity metric selection. Given that the largest data structure is selected, the longest distance in this example is stored at the root. The longest distance finder 114d may find and select the longest distance from nodes 0 to n. The longest distance finder 114d provides the longest distance to the first to Nth similarity PEs 110a to 110n. The longest distance may be the lowest similarity score (longest distance) from all similarity scores (distances) stored in the heap memory 114c.

[0033] Each distance in the heap memory 114c reflects the similarity between one of the candidate vectors V000-Vn03 and the first query vector 106a. In this example, a higher similarity corresponds to a shorter distance, while a lower similarity corresponds to a longer distance. Thus, the longest distance is the distance of the candidate vector among the candidate vectors V000-Vn03 identified by the heap memory 114c that has the smallest similarity to the first query vector 106a. The longest distance is used to perform parallel pruning to stop analyzing candidate vectors among the candidate vectors V000-Vn03 at an early stage.

[0034] For example, for a certain period of time, until the heap memory 114c is full, the first through Nth similarity PEs 110a through 110n may perform the entire distance calculation between the candidate vectors V000 through Vn03 and the first query vector 106a, and provide the total distance and the vector ID to the hardware heap engine 114. When the heap memory 114c is full, the first through Nth similarity PEs 110a through 110n may perform a partial pruning process to determine whether to prune the calculation for the candidate vectors V000 through Vn03 until the entire distance calculation is completed.

[0035] For example, assume that the longest distance finder 114d identifies that the longest distance stored in the heap memory 114c is “3.” Further assume that the first similarity PE 110a compares the first query vector 106a with the candidate vector V002. For example, the first query vector 106a may have 512 features, each of which is approximately 1 byte. Similarly, the candidate vector V002 may have 512 features, each of which is approximately 1 byte. The first similarity PE 110a may compare features at the same index (e.g., byte) location to determine how similar the features are to each other and generate a distance based on the similarity. The first similarity PE 110a may accumulate the distances of the vector features compared to date (e.g., a sum of the distances calculated to date, a running average of the distances calculated to date, a weighted sum of the distances calculated to date, etc.) to form a partial distance. The partial distance may be the distance accumulated in a running distance calculation. For example, if features at byte positions 0-3 of candidate vector V002 and features at byte positions 0-3 of first query vector 106a are compared to each other and have an associated distance, the partial distance will be the sum of the associated distances (e.g., partial distance = distance of feature at byte value 0 + distance of feature at byte value 1 + distance of feature at byte value 2 + distance of feature at byte value 3). It is worth noting that there may be 512 byte positions, and the partial distance reflects only the first part (first 4 bytes) of those 512 byte positions. Thus, the partial distance is the running total of all distances calculated between the features of candidate vector V002 and the features of first query vector 106a.

[0036] If the partial distance exceeds the longest distance received from the longest distance finder 114d, the first similarity PE 110a may stop determining the similarity between the candidate vector V002 and the first query vector 106a. For example, assume that the partial distance between the candidate vector V002 and the first query vector 106a has a value of 4 (accumulated over the first four bytes), while the longest distance has a value of 3. It can already be concluded that the candidate vector V002 is more dissimilar to the first query vector 106a than any of the candidate vectors V000-Vn03 already analyzed to have associated distances stored in the heap memory 114c. That is, the longest distance represents the highest dissimilarity to the first query vector 106a in the heap memory 114c, and analysis of any other candidate vector among the candidate vectors V000-Vn03 may be bypassed and ignored (pruned) if the partial distance of that other candidate vector exceeds the longest distance, regardless of how many of those other candidate vectors are analyzed.

[0037] Doing this may conserve processing power and reduce latency. In this example, candidate vector V002 has been analyzed for four byte positions and has already accumulated a partial distance that exceeds the longest distance. Therefore, it is reasonable to conclude that candidate vector V002 will not be a final similarity match for first query vector 106a. Therefore, the remaining bytes of candidate vector V002 do not need to be analyzed for similarity to first query vector 106a, and first similarity PE 110a discards further analysis of candidate vector V002 in favor of analyzing other candidate vectors among candidate vectors V000-Vn03.

[0038] Furthermore, the first through Nth similarity PEs 110a through 110n operate in a cyclical manner to avoid stalls and waits: if pruning occurs while calculating distances across features (e.g., byte positions) at index n for the previous candidate vector, the distance calculation for the next candidate vector starts at index n+1. This leverages the commutative and associative properties of distance calculations, which ensure that the calculated distances remain the same regardless of partial calculations starting from any feature index. For example, candidate vector V002 is pruned based on a comparison of the features of candidate vector V002 at byte positions 0 through 3 with the features of the first query vector 106a. Therefore, the first similarity PE 110a may have its index set to byte position 3. When the first similarity PE 110a starts comparing the first query vector 106a with candidate vector V003, the first similarity PE 110a does not need to reset its index to 0. Rather, the first similarity PE 110a compares the feature amount of the candidate vector V003 with the feature amount of the first query vector 106a at byte position 4 (index+1), which is the next byte position after the candidate vector V002 is discarded. When the last byte position is reached, the first similarity PE 110a may return to byte position 0 and determine the distance at byte positions 0 to 3 of the candidate vector V003.

[0039] The first similarity PE 110a may iterate through all byte positions (including byte positions 0-4) of the candidate vector V003 and the first query vector 106a, as long as the partial distance does not exceed the maximum distance. Assume that the partial distance does not exceed the maximum distance, so all 512 bytes of the candidate vector V003 are analyzed. The final distance may be the sum of all distances between the features of the first query vector 106a and the features of the candidate vector V003. That is, the features at all 512 byte positions of the first query vector 106a and the candidate vector V003 are compared to generate distances that are summed together to form the total distance. If the total distance is less than the maximum distance, the candidate vector V003 is determined to be in the list of the current top-K nearest neighbors for the first query vector 106a and provided to the hardware heap engine 114. Although byte positions are described above, some embodiments may work for different feature vector sizes (INT8, INT16, INT32, FP32, BF16) with different index positions.

[0040] The first similarity PE 110a may send the vector ID and total distance of candidate vector V003 to the hardware heap engine 114 for storage. The heap controller 114b may store the vector ID and total distance of candidate vector V003 in heap memory 114c and remove the vector ID associated with the longest distance and this longest distance. The longest distance finder 114d may select a new longest distance from heap memory 114c and propagate this new longest distance to the first through Nth similarity PEs 110a through 110n. In particular, because the calculation of candidate vector V002 has been pruned, the first similarity PE 110a does not send the partial distances and vector ID of candidate vector V002.

[0041] Therefore, each of the first through Nth similarity PEs 110a through 110n may perform a partial distance analysis by comparing the partial distances of the candidate vectors V000 through Vn03 with the maximum distance and stopping the analysis when the partial distance is greater than the maximum distance. After all of the candidate vectors V000 through Vn03 have been analyzed for similarity to the first query vector 106a, the architecture 100 may output the results to a user or store the results. For example, the shortest distances and corresponding node IDs may be identified in the heap memory 114c. A final vector of the candidate vectors V000 through Vn03 may be identified based on the corresponding node IDs and output as the closest match to the first query vector 106a. In some examples, an application may request a total of K nearest neighbor / closest candidate matches to the query vector, where the maximum value of K is the size of the heap memory 114c (e.g., the number of nodes 0 through n). The hardware heap engine 114 may then return the k distances and the corresponding node IDs to the application.

[0042] The heap controller 114b may then remove all nodes from the heap memory 114c. The scheduler 104 may propagate the second query vector 106b to the first through Nth similarity PEs 110a through 110n. The first through Nth similarity PEs 110a through 110n may analyze the second query vector 106b for similarity to the candidate vectors V000 through Vn03, as described above. After the similarity analysis of the second query vector 106b is completed (comparison with all candidate vectors V000 through Vn03 is completed), another query vector is sequentially streamed and analyzed until processing of the last Nth query vector 106n is completed. Notably, throughout the streaming of the query vectors 106, the candidate vectors V000 through Vn03 remain in the memory area 108, avoiding high-latency memory accesses.

[0043] The memory area 108 may be an embedded static random access memory (SRAM) embedded (e.g., on-chip) within the similarity search processor 116. It is also worth mentioning that variations on the heap memory 114c are possible (e.g., min-heap binary tree storage or max-heap binary tree storage).

[0044] 2A-2B, a far-memory similarity search and matching architecture 300 is disclosed. Architecture 300 operates similarly to architecture 100 described above, and similar features will not be described in detail for the sake of brevity. However, it will be understood that aspects of architecture 100 are readily incorporated into architecture 300.

[0045] As shown, a first processing array 324, a second processing array 326, and a third processing array 328 are provided. The first processing array 324, the second processing array 326, and the third processing array 328 may be located on the same SoC and / or form part of the same processor. While the first processing array 324 is shown in detail, it will be understood that the second and third processing arrays 326, 328 are comprised of similar features and elements that are not shown for the sake of brevity. The first processing array 324, the second processing array 326, and the third processing array 328 may process different queries in parallel with each other.

[0046] In this example, the candidate vectors cannot fit entirely within the first through Nth memory areas 310-314 and are therefore retrieved and removed as needed. In this example, a transit buffer 304 and a query buffer 306 receive both query and vector data from fabric 302, which may be a network-on-chip fabric. Transit buffer 304 is appropriately sized to buffer candidate vectors from a candidate vector database (which may be stored in off-chip storage).

[0047] Each of the similarity PEs 316 is connected to a dedicated memory area of ​​the memory area 310. Each of the memory areas 310 may operate as a "circular ping pong vector buffer" (CPPVB), storing two database vectors in first and second buffers. To operate as a CPPVB, the first and second buffers of the memory area 310 store multiple vectors at a time. For example, assume that the first similarity PE 316a has completed processing a candidate vector in one of the first and second buffers of the first memory area 310a. That one buffer is replenished from the row vector buffer 334, while the first similarity PE 316a processes the candidate vector in the other of the first and second buffers.

[0048] Similarity PEs 316 are connected in a 1D systolic array fashion. Results from similarity PEs 316 are daisy-chained through result engine 318 to form a single stream of results that are sent to multiplexer (MUX) 320a. MUX 320a may provide the results to MUX 320b, which provides the results to shared hardware heap engine (SHHE) 336. SHHE 336 is discussed in more detail with respect to Figure 2B, which shows SHHE 336 in more detail.

[0049] Multiple similarity PEs 316 may compute similarity searches for the same query (e.g., the same query vector). The query streaming mechanism of the first query buffer 306a is the same as that of the query buffer 102 (FIG. 1) and will not be repeated in detail. As mentioned above, each of the similarity PEs 316 is connected to one of the first through Nth memory areas 310a through 310n, where the one memory area includes a CPP VPB. For example, the first memory area 310a includes a CPP VPB that includes first and second buffers. The first and second buffers store two vectors from a database for comparison with the query. Each feature in the vectors stored in the first and second buffers is repeatedly streamed to the first similarity PE 316a.

[0050] For example, initially, all similarity PEs 316 begin calculations on the first feature in their associated first buffers. At each clock cycle, the similarity PEs 316 receive the buffer stream from the first buffer and determine the similarity between the "i" feature of the candidate vector and the query vector to determine the partial distance. When the candidate vector calculation is pruned, the query stream index is not interrupted. For example, the SHHE 336 may provide the maximum distance. Assume that the first similarity PE 316a determines that, at feature index "i" (e.g., byte position), the first candidate vector in the first buffer of the first memory area 310a is to be discarded (e.g., based on the partial distance being greater than the maximum distance). The first similarity PE 316a accesses the second buffer in the first memory area 310a and retrieves the second candidate vector from the second buffer without interrupting the flow of query bytes being streamed. The first similarity PE 316a analyzes the similarity between the second candidate vector and the query vector starting from feature index "i+1" and continues in a circular manner until all features have been processed or the calculation of the second vector is pruned.

[0051] While the first similarity PE 316a performs similarity analysis based on the second candidate vector, the row vector buffer 334 may store the third candidate vector in the first buffer of the first memory area 310a. After processing of the second vector is completed or calculation of the second vector is pruned, the first similarity PE 316a switches to the first buffer and begins similarity analysis for the third candidate vector. The row vector buffer 334 may begin storing the fourth candidate vector for the first similarity PE 316a in the second buffer for analysis. Thus, the first similarity PE 316a ping-pongs between the first buffer and the second buffer. The second through Nth similarity PEs 316b through 316n may similarly access candidate vectors from the first and second buffers of the second through Nth memory areas 310b through 310n in a ping-pong fashion.

[0052] The transit buffer 304 fetches a vector database (e.g., 1 billion candidate vectors) from system memory (not shown). The candidate vectors in the transit buffer 304 are broadcast to all row vector buffers of the first, second, and third processing arrays 324, 326, and 328, including the row vector buffer 334. The first and second buffers fetch candidate vectors from the row vector buffer 334. Since the first through Nth similarity PEs 316a through 316n calculate similarity measures for the same query, the candidate vectors stored in the first and second buffers are different and mutually exclusive. It is worth noting that each of the first, second, and third processing arrays 324, 326, and 328 may store the same candidate vector.

[0053] For example, each of the first, second, and third processing arrays 324, 326, and 328 may receive a first candidate vector and perform a similarity analysis on the first candidate vector. When all of the first, second, and third processing arrays 324, 326, and 328 have received the first candidate vector, the transit buffer 304 may remove the first candidate vector from its memory and replace it with a new vector (e.g., a second candidate vector) from the fabric 302. Row vector buffers, including row vector buffer 334, may then provide the new vector to the first, second, and third processing arrays. Vectors may be fetched by the transit buffer 304 and pulled by row vector buffers, such as row vector buffer 334, continuously until the entire database has been analyzed for similarity.

[0054] As described above, the second and third processing arrays 326, 328 are composed of similar components as the first processing array 324. The second query buffer 306b may provide queries to the second processing array 326. The third query buffer 306c may provide queries to the third processing array 328.

[0055] In the architecture 300, the first, second, and third processing arrays 324, 326, and 328 perform similarity searches in parallel across multiple different queries. The SHHE 336 operates similarly to the hardware heap engine 114 (FIG. 1), but with the additional ability to handle data associated with several separate query searches and perform pruning based on partial distances, as described above. Results from each of the first through Nth similarity PEs 316a-316n are daisy-chained through the first through Nth result engines 318a-318n. Each of the first through Nth result engines 318a-318n may include two buffers for storing results in case of backpressure or delays in sending results through the first through Nth result engines 318a-318n.

[0056] The outputs from the first, second, and third processing arrays 324, 326, 328 are provided to MUXes 320a, 320b to generate a single stream of results from the first, second, and third processing arrays 324, 326, 328. Figure 2B shows a more detailed view of the SHHE 336, with the relevant components from Figure 2A also shown. Referring now to Figure 2B, to avoid collisions and backpressure for writes to the SHHE 336, the result stream may be stored in a large size buffer 332 (e.g., a 64-deep elastic buffer). A heap controller 330 reads the candidate vector IDs, corresponding query IDs, and corresponding distances stored in buffer 332 and controls the flow of insertions into the corresponding partition for the query ID.

[0057] For example, the first, second, and third processing arrays 324, 326, and 328 may provide outputs including a candidate vector ID corresponding to a candidate vector compared against a query vector, a total distance associated with the candidate vector and the query vector, and a query ID corresponding to the query vector. The query ID may be referenced to determine whether to store the candidate vector and the corresponding total distance in the first heap memory 322a, the second heap memory 322b, or the third heap memory 322c of the heap memories 322. The first heap memory 322a may store results from the first processing array 324 associated with a first query in nodes 0-n. The second heap memory 322b may store results from the second processing array 326 associated with a second query in nodes 0-n. The third heap memory 322c may store results from the third processing array 328 associated with a third query in nodes 0-n.

[0058] For example, assume that the second processing array 326 analyzes a first candidate vector for similarity to a second query vector. The second processing array 326 may determine that the total distance of the first candidate vector is less than the longest distance associated with the second query vector. Therefore, the second processing array 326 may provide an output to the MUX 320a that includes the first candidate vector ID, the second query ID (associated with the second query vector), and the total distance. The MUXes 320a and 320b may provide this output to the SHHE 336. The SHHE 336 may receive this output and store it in the buffer 332 until the heap controller 330 is ready to store it. The heap controller 330 may receive this output and extract the second query ID. The second query ID may correspond to the second heap memory 322b. That is, each result associated with the second query vector may be stored in the second heap memory 322b. Therefore, the heap controller 330 may associate the first candidate vector ID and the distance with each other and store them in the second heap memory 322b. Therefore, the heap controller 330 may identify the query ID to determine where to store the candidate vector ID and the distance.

[0059] The longest distance finder 338 further determines the longest distance from the first heap memory 322a. The longest distance from the first heap memory 322a may be the longest distance of the first query stored in the first heap memory 322a. The longest distance finder 338 further provides the longest distance of the first query to the first processing array 324. The longest distance finder 338 further determines the longest distance from the second heap memory 322b. The longest distance from the second heap memory 322b may be the longest distance of the second query stored in the second heap memory 322b. The longest distance finder 338 further provides the longest distance of the second query to the second processing array 326. The longest distance finder 338 further determines the longest distance from the third heap memory 322c. The longest distance of the third heap memory 322c may be the longest distance of the third query stored in the third heap memory 322c. The longest distance finder 338 further provides the longest distance of the third query to the third processing array 328. The first processing array 324 may perform a pruning process based on the longest distance of the first query. Similarly, the second and third processing arrays 326, 328 may perform pruning processes based on the longest distances of the second and third queries, respectively.

[0060] 3 illustrates a method 800 of a similarity search process with pruning. Method 800 may be implemented generally using embodiments described herein, such as architecture 100 (FIG. 1) and / or architecture 300 (FIGS. 2A-2B) previously discussed. In one embodiment, method 800 is implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium, such as random access memory (RAM), read-only memory (ROM), programmable ROM (PROM), firmware, flash memory, etc.; in configurable logic, such as a programmable logic array (PLA), field programmable gate array (FPGA), complex programmable logic device (CPLD), etc.; in fixed-function logic hardware, using circuit technology, such as an application-specific integrated circuit (ASIC), complementary metal-oxide semiconductor (CMOS), or transistor-transistor logic (TTL) technology, etc., or any combination thereof.

[0061] For example, computer program code for performing the operations illustrated in method 800 may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java®, SMALLTALK®, C++, etc., and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. Additionally, logic instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, state setting data, configuration data for integrated circuits, state information for personalizing electronic circuits, and / or other structural components specific to hardware (e.g., a host processor, central processing unit / CPU, microcontroller, etc.).

[0062] Illustrated processing block 802 determines, using a first processing engine of the plurality of processing engines, a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector. Illustrated processing block 804 determines, using a second processing engine of the plurality of processing engines, a total similarity measure based on the query vector and a second candidate vector. Illustrated processing block 806 determines, using the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure.

[0063] In some embodiments, method 800 further comprises, in response to the first partial similarity measure being less than the total similarity measure, comparing, using a first processing engine, a second portion of the query vector with a second portion of the first candidate vector, where the first partial similarity measure is a partial distance and the total similarity measure is a total distance. In some examples, method 800 further comprises: retrieving, using a plurality of processing engines, different candidate vectors; determining, using the plurality of processing engines, a plurality of partial similarity measures between the first portion of the query vector and the first portion of the different candidate vector; and determining, using the plurality of processing engines, whether to bypass the partial similarity calculation between the second portion of the query vector and the second portion of the different candidate vector based on the plurality of partial similarity measures and the total similarity measure.

[0064] In some examples, the method 800 further includes accessing multiple memory storage areas, each dedicated to one of the multiple processing engines. The multiple memory storage areas store different candidate vectors. The different candidate vectors represent a vector candidate database. In some examples, the method 800 further includes determining, using a first processing engine, to bypass a similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure. The index for the query vector is a value at which the first partial similarity measure is determined. In response to bypassing the similarity calculation for the first candidate vector, the method 800 increments, using the first processing engine, a value of the index and determines, using the first processing engine, whether to bypass a similarity calculation for the third candidate vector based on the partial similarity measure to be determined based on the feature value of the third candidate vector and the feature value of the query vector, wherein both the feature value of the third candidate vector and the feature value of the query vector are associated with the incremented value of the index.

[0065] In some examples, the method 800 further includes storing the total similarity measure and the plurality of similarity measures in a max-heap binary tree or a min-heap binary tree. The plurality of similarity measures are determined based on different candidate vectors and the query vector. The total similarity measure is greater than each of the plurality of similarity measures. In some examples, the method 800 further includes storing the plurality of candidate vectors in a plurality of ping-pong buffers; determining, using a plurality of processing engines, a plurality of partial similarity measures based on a first portion of the plurality of query vectors and a first portion of the plurality of candidate vectors; and determining, using the plurality of processing engines, a similarity calculation associated with a first subset of the plurality of candidate vectors to be bypassed based on the first subset of the plurality of partial similarity measures and the first total similarity measure.

[0066] In some examples, the method 800 further includes determining, using a group of multiple processing engines, that a second subset of the plurality of candidate vectors is to be processed based on the second subset of the plurality of partial similarity measures and the first total similarity measure. The method 800 further includes determining, using the group of multiple processing engines, a second total similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors, and storing each respective total similarity measure of the second total similarity measures in a different heap memory based on an identity of one of the plurality of query vectors associated with the respective total similarity measure. Each of the different heap memories is dedicated to one of the plurality of query vectors.

[0067] FIG. 4 shows a timing diagram 400 for PE0 402 and PE0n 404. PE0 402 and PE0n 404 can easily be substituted for either the similarity PE 110 (FIG. 1) or the similarity PE 316 (FIGS. 2A-2B). PE0 402 and PE0n 404 operate on a query having 512 features, each 1 byte in size. The 512 bytes in the query are represented as Q0, Q1...Q511 in the timing diagram 400. The diagram presents two cases: pruning the computation for PE0 402 and accepting the computation for PE0n 404.

[0068] At the start of timing diagram 400, referring to signal 406, PE0 402 compares query vector Q with candidate vector V00 (indicated in the Buffer / Vector Number row) from memory bank 0. At the 226th clock cycle (performed at Q226), the accumulated partial distance 226 (hatched in background) for v00 is greater than the maximum distance (which may be received from the hardware heap engine or SHHE), and the similarity calculation for V00 is pruned (e.g., terminated) at index 226. At the next clock cycle, candidate vector v01 is loaded into PE0 402, and Q227 from the query vector is used to calculate the similarity distance from the 227th feature from candidate vector v01 (V227 at the index +1 position). Therefore, PE0 402 begins the distance calculation for the next database candidate vector v01. PE0 402 continues the similarity calculation up to index 511 and then cyclically down to indexes 0, 1...226 unless pruning occurs to drop the calculation.

[0069] The query stream is shared between PE0 402 and PE0n 404. Therefore, both PE0 402 and PE0n 404 operate on the same query vector. In the lower signal 408 of the timing diagram 400, PE0n 404 compares the query byte Q0 with the candidate vector vn0 with an offset of n clock cycles relative to PE0 402, spanning the time difference between time B and time A. This is due in part to the daisy-chain transmission of the query vector throughout the PEs, including PE0 402 and PE0n 404. Therefore, PE0n 404 receives the query vector after PE0 402. PE0n 404 calculates the partial distances for the query features and the corresponding vector features of the candidate vector vn0 over 512 clock cycles, not exceeding the maximum distance. After the 512th clock cycle (511th clock cycle for 0-indexed), the partial distance is still less than the maximum distance. Therefore, the candidate vector vn0 is qualified as a K-nearest neighbor candidate for the query vector and is consequently pushed to the hardware heap engine or SHHE for storage. In the next clock cycle, PE0n 404 selects feature Q0 from the query vector. PE0n 404 selects feature 0 (V0) from the new candidate vector vn1 and begins calculating the partial distance.

[0070] FIG. 5 illustrates a query streaming method 500. Method 500 may generally be implemented using embodiments described herein, such as architecture 100 (FIG. 1), architecture 300 (FIGS. 2A-2B), and / or timing diagram 400 (FIG. 4) previously discussed. For example, method 500 may be performed by scheduler 104 of query buffer 102 (FIG. 1) and / or query buffer 306 (FIGS. 2A-2B). Method 500 may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc.; in configurable logic such as a PLA, FPGA, CPLD, etc.; in fixed-function logic hardware using circuit technologies such as ASIC, CMOS, or TTL technology, or any combination thereof.

[0071] Illustrated processing block 502 streams the selected query vector to multiple similarity PEs. Illustrated processing block 504 determines whether the selected query vector has been compared against all candidate vectors. If not, processing block 502 is executed. If so, illustrated processing block 506 determines whether all query vectors are complete. If not, illustrated processing block 508 selects a new query vector as the selected query vector, and processing block 502 is executed to process the selected query vector and calculate a similarity measure of the selected query vector against the candidate vectors.

[0072] 6 illustrates a similarity calculation method 530 implemented by a similarity PE. Method 530 may generally be implemented using embodiments described herein, such as architecture 100 (FIG. 1), architecture 300 (FIGS. 2A-2B), timing diagram 400 (FIG. 4), and / or method 500 (FIG. 5) previously discussed. For example, method 530 may be performed by similarity PE 110 (FIG. 1) and / or similarity PE 316 (FIG. 2A). Method 530 may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium, such as RAM, ROM, PROM, firmware, flash memory, etc.; in configurable logic, such as a PLA, FPGA, CPLD, etc.; in fixed-function logic hardware, using circuit technology, such as ASIC, CMOS, or TTL technology, etc., or any combination thereof.

[0073] The illustrated processing block 542 sets an index value to 0. The index value may be a byte position (or correspond to the feature vector size of the query and candidate vectors) that the similarity PE will refer to to compare the feature values ​​of the candidate vector and the query vector at that byte position and determine the similarity measure. That is, the similarity PE initially starts fetching address 0 for the candidate and query vectors. Therefore, initially, both the query vector and the candidate vector start at index 0.

[0074] Illustrated processing block 532 calculates a feature distance for the query vector features at the index value and the candidate vector features at the index value. Illustrated processing block 534 adds the feature distance to the partial distance to generate a sum and sets this sum as the new partial distance. In some examples, processing block 534 calculates an average of the distances calculated so far or a weighted sum of the distances calculated so far and sets this value as the partial distance. Illustrated processing block 536 determines whether the partial distance is greater than the maximum distance.

[0075] If the partial distance is greater than the maximum distance, the remainder of the calculations are pruned and discarded for that candidate vector. For example, illustrated processing block 544 determines whether any more candidate vectors exist. If so, illustrated processing block 546 selects a new candidate vector from the remaining candidate vectors and sets the partial distance to 0. Processing block 548 increments the index value. Thereafter, processing block 532 is executed.

[0076] If processing block 536 determines that the maximum distance is greater than the partial distance, illustrated processing block 538 determines whether the last feature in the candidate vector has been reached. If not, illustrated processing block 540 increments the index value, so that processing block 532 calculates the vector distance of the feature at the incremented index value, and so on. If processing block 538 determines that the last feature in the candidate vector has been reached, illustrated processing block 542 pushes the results to a sorting engine (e.g., a hardware heap engine or SHHE). That is, once the partial distances for all features have been accumulated and the final total distance (which is the accumulation of all partial distances) is still less than the maximum distance, the results are sent to the sorting engine.

[0077] 7 illustrates a heap memory structure 550 that is a binary tree. Heap memory structure 550 may generally be implemented using embodiments described herein, such as architecture 100 (FIG. 1), architecture 300 (FIGS. 2A-2B), timing diagram 400 (FIG. 4), method 500 (FIG. 5), and / or method 600 (FIG. 6) previously discussed. For example, nodes 0-n of heap memory 114c (FIG. 1), nodes 0-n of first heap memory 322a (FIG. 2B), nodes 0-n of second heap memory 322b (FIG. 2B), and / or nodes 0-n of third heap memory 322c (FIG. 2B) may be organized into heap memory structure 550.

[0078] The heap memory structure 550 may include nodes 1-15 organized in a heap structure, numbered (e.g., pre-order sequencing) from root (node ​​1) to leaves (nodes 8-15). The node numbering corresponds to storage locations (node ​​indexes) within the hardware heap engine. The hardware heap engine partitions common memory to store K nearest neighbors (KNN) for a batch of queries, where K and the batch size are configurable. The number of nodes (15) is exemplary; embodiments as described herein may include any number of nodes and may determine for multiple KNN values ​​to be stored (e.g., 20 KNN values ​​would result in 20 nodes).

[0079] In structure 550, the heap memory structure 550 is configured to store the 15 closest vectors in each partition for a query. The heap binary structure may be a max-heap binary tree in which the root node 1 has the largest distance value, the first tier (i.e., nodes 2 and 3) has the next largest distance value, the second tier (i.e., nodes 4-7) has the next largest distance value, and the bottom tier (i.e., nodes 8-15) has the lowest distance value. As will be explained in more detail, a max-heap binary tree can be an efficient storage structure because the longest distance is always maintained and the root of the tree can be easily identified.

[0080] Furthermore, the insertion of new values ​​into the tree can be performed efficiently. For example, when a new distance value is to be inserted into structure 550, the distance value in node 1 (the longest distance) is automatically removed. The new distance value may be compared with the distance value in node 2. If the distance heap value of node 2 is greater than the new distance value, the distance value in node 2 (and the corresponding candidate vector ID) may be moved to node 1, and the new distance value may occupy node 2. The new distance value is then compared with the distance of one of node 2's child nodes (nodes 4 and 5), and if the new distance is less than the distance of the one child node, it is swapped with the one child node. This process may be repeated until the new distance is no longer less than the child node of the currently occupied node of the new distance, or until the new distance position is at the bottom of the max heap binary tree. Notably, the new distance does not need to be compared with all the distances of nodes 2 through 15; only three comparisons (at most) need to be performed to find the final position. That is, a strict ordering of the distances from largest to smallest is not required; only the largest distance needs to be identified, which is stored at node 1. Additionally, each of nodes 1-15 may include a candidate vector ID corresponding to the distance value stored at the respective node (e.g., the candidate vector ID of the candidate vector that underwent the similarity calculation process to generate the distance value stored at the node).

[0081] Structure 550 may be replicated per partition, query memory, or query. A copy of the root of node 1 is stored in a register in hardware and broadcast as the maximum distance to the appropriate similarity PEs operating on each query for comparison and elimination of redundant results.

[0082] In some examples, if the distance calculation is not pruned / dropped, the distance result is daisy-chained to a hardware heap engine (HHE), which creates structure 550. The HHE is a device for hardware-friendly implementation of a traditional heap. A heap, particularly a max heap (or max-heap) binary tree, efficiently stores distances for k "nodes" and can easily access the maximum distance from the root node 1. A property of a max heap is that the value in a node must be greater than its child nodes; conversely, for a min heap, the value of a node must be less than its child nodes. Therefore, the root node 1 of structure 550 stores the largest element in data structure 550. The HHE may be configured to run as a max heap or a min heap in some embodiments.

[0083] 8 illustrates a method 420 implemented by an HHE and / or SHHE to fill an incomplete (not yet fully filled) structure (e.g., a binary tree). Method 420 may generally be implemented using embodiments described herein, such as architecture 100 (FIG. 1), architecture 300 (FIGS. 2A-2B), timing diagram 400 (FIG. 4), method 500 (FIG. 5), method 530 (FIG. 6), and / or structure 550 (FIG. 7) previously discussed. For example, method 420 may generate structure 550 (FIG. 7). Method 420 may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc.; in configurable logic such as a PLA, FPGA, CPLD, etc.; in fixed-function logic hardware using circuit technology such as ASIC, CMOS, or TTL technology, etc., or any combination thereof.

[0084] Illustrated processing block 422 identifies a new node entry. Illustrated processing block 424 stores the new node entry in the first available location, starting from index 1 (e.g., downward from the root node to lower levels). Illustrated processing block 426 determines whether the new node location is the root node. If so, no further action is required. If the current location is a non-root node, the current location is a child node. Therefore, illustrated processing block 428 determines whether the distance value of the new node entry (stored in the child node) is greater than the distance of the parent node of the new node location (child node). If not, no further action is required. If so, and the distance of the current node is greater than the distance of the parent node, illustrated processing block 430 moves the new node entry to the parent node and moves the parent node entry to the child node. That is, illustrated processing block 430 swaps the parent's node entry with the new node entry in the child node. Thereafter, illustrated process block 426 is executed again with the new node's location set to the parent node location.

[0085] 9 illustrates a method 440 implemented by an HHE and / or SHHE to insert a new node entry into a full structure (e.g., a binary tree with all nodes populated). Method 440 may generally be implemented using embodiments described herein, such as architecture 100 (FIG. 1), architecture 300 (FIGS. 2A-2B), timing diagram 400 (FIG. 4), method 500 (FIG. 5), method 530 (FIG. 6), structure 550 (FIG. 7), and / or method 420 (FIG. 8) previously discussed. For example, method 440 may update structure 550 (FIG. 7). The method 440 may be implemented in one or more modules as a set of logic instructions stored in a machine or computer readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc.; in configurable logic such as, for example, a PLA, FPGA, CPLD, etc.; in fixed function logic hardware using circuit technologies such as, for example, ASIC, CMOS or TTL technology, or any combination thereof.

[0086] Illustrated processing block 442 inserts a new node entry (with a total distance less than the longest distance of the head or root node) into the head node of the max heap binary tree. The entry previously in the head of the max heap binary tree is deleted and removed from the max heap binary tree. Illustrated processing block 444 reads the distances of the left and right child nodes of the parent node. Illustrated processing block 446 determines whether the distance of the right child node is greater than the distance of the left child node. If so, illustrated processing block 454 determines whether the distance of the right child node is greater than the distance of the parent node. If not, method 440 ends. If processing block 454 determines that the distance of the right child node is greater than the distance of the parent node, illustrated processing block 450 exchanges the node entry of the parent node with the entry in the right child node. Illustrated processing block 452 determines whether the right child node is a leaf node (the lowest level of the binary tree). If so, method 440 ends. If not, process block 460 is illustrated as setting the right child node to the parent node and process block 444 is executed.

[0087] If processing block 446 determines that the distance of the right child node is not greater than the distance of the left child node, illustrated processing block 448 is executed. Processing block 448 determines whether the distance of the left child node is greater than the distance of the parent node. If not, method 440 ends. If processing block 448 determines that the distance of the left child node is greater than the distance of the parent node, illustrated processing block 456 exchanges the node entry of the parent node with the entry in the left child node. Illustrated processing block 458 determines whether the left child node is a leaf node. If not, illustrated processing block 462 sets the left child node to the parent node. If it is a leaf node, method 440 ends.

[0088] 10 , a similarity search and pruning query processing computing system 158 is shown. System 158 may generally be part of an electronic device / platform having computing capabilities (e.g., personal digital assistant / PDA, notebook computer, tablet computer, convertible tablet, server), communication capabilities (e.g., smartphone), imaging capabilities (e.g., camera, camcorder), media playback capabilities (e.g., smart television / TV), wearable capabilities (e.g., watch, eyeglasses, headwear, footwear, jewelry), vehicle capabilities (e.g., car, truck, motorcycle), robotic capabilities (e.g., autonomous robot), etc., or any combination thereof. In the illustrated example, system 158 includes a host processor 160 (e.g., CPU) having an integrated memory controller (IMC) 154 coupled to a system memory 164.

[0089] The illustrated system 158 also includes an input / output (IO) module 166 implemented as a system-on-chip (SoC) on a semiconductor die 170 along with a host processor 160, a graphics processor 162 (e.g., GPU), a similarity search processor 150, a ROM 140, and an AI accelerator 148. The illustrated IO module 166 communicates with, for example, a display 172 (e.g., a touchscreen, a liquid crystal display / LCD, a light-emitting diode / LED display), a network controller 174 (e.g., wired and / or wireless), an FPGA 178, and mass storage 168 (e.g., a hard disk drive / HDD, an optical disk, a solid-state drive / SSD, flash memory). Additionally, the SoC 170 may further include a processor (not shown) dedicated to artificial intelligence (AI) and / or neural network (NN) processing and / or the AI ​​accelerator 148. For example, system SoC 170 may include a visual processing unit (VPU) and / or other AI / NN-specific processors, such as AI accelerator 148. In some embodiments, any aspect of the embodiments described herein may be implemented in a processor and / or accelerator dedicated to AI and / or NN processing, such as AI accelerator 148, graphics processor 162, host processor 160, and / or similarity search processor 150.

[0090] The similarity search processor 150 may execute instructions 156 retrieved from a system memory 164 (e.g., dynamic random access memory) and / or mass storage 168 to implement aspects as described herein. The similarity search processor 150 includes PE1 to PE2 that perform batch processing, similarity search of candidate vectors to a query vector, and early pruning of candidate vector calculations. n To do this, some examples may include PE1 to PE nThe candidate vectors may be stored in a memory storage area 144 having multiple partitions, each dedicated to one of the memory storage areas 144. If the candidate vectors are too large to fit within the memory storage area 144, a subset of the candidate vectors may be stored in a memory storage area 144, each dedicated to one of the memory storage areas 144. n The query vector may be stored in a ping-pong buffer 142 (e.g., static random access memory) that the processor 152 accesses to compare the query vector to the subset of candidate vectors. The query and candidate vectors may be stored in mass storage 168 when not in use and moved to memory storage area 144, ping-pong buffer 142, and / or system memory 164 when a similarity search is to be performed. When instructions 156 are executed, computing system 158 may implement one or more aspects of the embodiments described herein. For example, system 158 may implement one or more aspects of architecture 100 (FIG. 1), architecture 300 (FIGS. 2A-2B), timing diagram 400 (FIG. 4), method 500 (FIG. 5), method 530 (FIG. 6), structure 550 (FIG. 7), method 420 (FIG. 8), and / or method 440 (FIG. 9) previously discussed. Thus, the illustrated computing system 158 is considered to have improved performance at least to the extent that it enables the computing system 158 to utilize a low latency similarity search and pruning process to reduce processing power, overhead, and far memory accesses. In some examples, the memory storage area 144 may operate and include the ping-pong buffer 142, if desired.

[0091] 11 illustrates a semiconductor device 180 (e.g., a chip, die, package). The illustrated device 180 includes one or more substrates 184 (e.g., silicon, sapphire, gallium arsenide) and logic 182 (e.g., transistor arrays and other integrated circuit / IC components) coupled to the substrate 184. In one embodiment, the device 180 is operated during an application development phase, and the logic 182 performs one or more aspects of the previously discussed architecture 100 ( FIG. 1 ), architecture 300 ( FIGS. 2A-2B ), timing diagram 400 ( FIG. 4 ), method 500 ( FIG. 5 ), method 530 ( FIG. 6 ), structure 550 ( FIG. 7 ), method 420 ( FIG. 8 ), and / or method 440 ( FIG. 9 ). Thus, logic 182 may use a first processing element of the plurality of processing elements to determine a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector, use a second processing element of the plurality of processing elements to determine a total similarity measure based on the query vector and the second candidate vector, and use the first processing element to determine whether to compare a second portion of the query vector with a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure. Furthermore, logic 182 may further include a processor (not shown) and / or AI accelerator dedicated to artificial intelligence (AI) and / or NN processing. For example, system logic 182 may include a VPU and / or other AI / NN-specific processor, e.g., an AI accelerator, a similarity search PE, etc. In some embodiments, any aspect of the embodiments described herein may be implemented in a processor and / or accelerator dedicated to AI and / or NN processing, e.g., an AI accelerator.

[0092] Logic 182 may be implemented at least in part in configurable logic or fixed-function hardware logic. In one example, logic 182 includes transistor channel regions positioned (e.g., buried) within substrate 184. Therefore, the interface between logic 182 and substrate 184 need not be an abrupt junction. Logic 182 may also be considered to include an epitaxial layer grown on an initial wafer of substrate 184.

[0093] Figure 12 illustrates a processor core 200 according to one embodiment. Processor core 200 may be the core of any type of processor, such as a microprocessor, embedded processor, digital signal processor (DSP), network processor, or other device that executes code. Although only one processor core 200 is shown in Figure 12, a processing element may alternatively include more than one of the processor cores 200 illustrated in Figure 12. Processor core 200 may be a single-threaded core, or, for at least one embodiment, processor core 200 may be multi-threaded in that it may include more than one hardware thread context (or "logical processor") per core.

[0094] FIG. 12 also illustrates memory 270 coupled to processor core 200. Memory 270 may be any of a wide variety of memories (including various layers of a memory hierarchy) known or otherwise available to those skilled in the art. Memory 270 includes one or more code 213 instructions executed by processor core 200. Code 213 may implement one or more aspects of embodiments, such as architecture 100 (FIG. 1), architecture 300 (FIGS. 2A-2B), timing diagram 400 (FIG. 4), method 500 (FIG. 5), method 530 (FIG. 6), structure 550 (FIG. 7), method 420 (FIG. 8), and / or method 440 (FIG. 9), previously discussed. Processor core 200 follows a program sequence of instructions represented by code 213. Each instruction may be input to front end 210 and processed by one or more decoders 220. The decoder 220 may generate as its output micro-operations, such as fixed-width micro-operations in a predefined format, or may generate other instructions, micro-instructions, or control signals that reflect the original code instructions. The illustrated front end 210 also includes register renaming logic 225 and scheduling logic 230, which generally allocate resources and queue operations corresponding to the conversion instructions for execution.

[0095] Processor core 200 is shown as including execution logic 250 having a set of execution units 255-1 through 255-N. Some embodiments may include multiple execution units dedicated to a particular function or set of functions. Other embodiments may include only one execution unit, or may include one execution unit capable of performing a particular function. The illustrated execution logic 250 performs operations specified by code instructions.

[0096] After completing execution of the operations specified by the code instructions, back-end logic 260 retires the instructions of code 213. In one embodiment, processor core 200 allows out-of-order execution but requires in-order retirement of instructions. Retirement logic 265 may take a variety of forms known to those skilled in the art (e.g., a reorder buffer, etc.). In this manner, processor core 200 is transformed during execution of code 213 in terms of, at least, the outputs generated by the decoders, the hardware registers and tables utilized by register renaming logic 225, and any registers (not shown) modified by execution logic 250.

[0097] 12, a processing element may include other elements on a chip with processor core 200. For example, a processing element may include memory control logic along with processor core 200. A processing element may include I / O control logic and / or I / O control logic integrated with memory control logic. A processing element may also include one or more caches.

[0098] 13, there is shown a block diagram of an embodiment of a computing system 1000 according to one embodiment. Illustrated in FIG. 13 is a multiprocessor system 1000 comprising a first processing element 1070 and a second processing element 1080. While two processing elements 1070 and 1080 are shown, it will be understood that an embodiment of system 1000 may also include only one such processing element.

[0099] System 1000 is shown as a point-to-point interconnect system, with a first processing element 1070 and a second processing element 1080 coupled via a point-to-point interconnect 1050. It should be understood that any or all of the interconnects shown in Figure 13 may be implemented as multi-drop buses rather than point-to-point interconnects.

[0100] 13, each of the processing elements 1070 and 1080 may be a multi-core processor including a first processor core and a second processor core (i.e., processor cores 1074a and 1074b, and processor cores 1084a and 1084b). Such cores 1074a, 1074b, 1084a, 1084b may be configured to execute instruction code in a manner similar to that discussed above in connection with FIG.

[0101] Each processing element 1070, 1080 may include at least one shared cache 1896a, 1896b. The shared cache 1896a, 1896b may store data (e.g., instructions) used by one or more components of the processor, such as cores 1074a, 1074b and 1084a, 1084b, respectively. For example, the shared cache 1896a, 1896b may locally cache data stored in memory 1032, 1034 for faster access by the components of the processor. In one or more embodiments, the shared cache 1896a, 1896b may include one or more intermediate level caches, such as a level 2 (L2), level 3 (L3), level 4 (L4), or other level cache, a last level cache (LLC), and / or a combination thereof.

[0102] While only two processing elements 1070, 1080 are shown, it will be understood that the scope of the embodiments is not so limited. In other embodiments, one or more additional processing elements may be present within a given processor. Alternatively, one or more of the processing elements 1070, 1080 may be elements other than processors, such as accelerators or field programmable gate arrays. For example, the additional processing elements may include an additional processor that is the same as the first processor 1070, an additional processor that is heterogeneous or asymmetric with the first processor 1070, an accelerator (e.g., a graphics accelerator or digital signal processing (DSP) unit), a field programmable gate array, or any other processing element. Various differences may exist between the processing elements 1070, 1080 in terms of various metrics of merit, including architectural characteristics, microarchitectural characteristics, thermal characteristics, power consumption characteristics, etc. These differences may effectively manifest as asymmetries and heterogeneities between the processing elements 1070, 1080. For at least one embodiment, the various processing elements 1070, 1080 may reside within the same die package.

[0103] The first processing element 1070 may further include memory controller logic (MC) 1072 and point-to-point (PP) interfaces 1076 and 1078. Similarly, the second processing element 1080 may include MC 1082 and PP interfaces 1086 and 1088. As shown in FIG. 13 , MCs 1072 and 1082 couple the processors to respective memories, i.e., memory 1032 and memory 1034, which may be part of main memory locally attached to the respective processors. While MCs 1072 and 1082 are shown as being integrated into the processing elements 1070, 1080, for alternative embodiments, the MC logic may be separate logic external to the processing elements 1070, 1080 rather than being integrated therewith.

[0104] First processing element 1070 and second processing element 1080 may be coupled to I / O subsystem 1090 via PP interconnects 1076 and 1086, respectively. As shown in FIG. 13 , I / O subsystem 1090 includes PP interfaces 1094 and 1098. Additionally, I / O subsystem 1090 includes interface 1092 that couples I / O subsystem 1090 with high-performance graphics engine 1038. In one embodiment, bus 1049 may be used to couple graphics engine 1038 to I / O subsystem 1090. Alternatively, a point-to-point interconnect may couple these components.

[0105] Additionally, I / O subsystem 1090 may be coupled to a first bus 1016 via an interface 1096. In one embodiment, first bus 1016 may be a bus such as a Peripheral Component Interconnect (PCI) bus, or a PCI Express bus, or another third generation I / O interconnect bus, although the scope of the embodiment is not limited thereto.

[0106] 13, various I / O devices 1014 (e.g., biometric scanner, speaker, camera, sensor) may be coupled to a first bus 1016, along with a bus bridge 1018 that may couple the first bus 1016 to a second bus 1020. In one embodiment, the second bus 1020 may be a low pin count (LPC) bus. In one embodiment, various devices may be coupled to the second bus 1020, including, for example, a keyboard / mouse 1012, a communication device 1026, and a data storage unit 1019, such as a disk drive or other mass storage device, that may include code 1030. The illustrated code 1030 may implement, for example, one or more aspects of previously discussed architecture 100 (FIG. 1), architecture 300 (FIGS. 2A-2B), timing diagram 400 (FIG. 4), method 500 (FIG. 5), method 530 (FIG. 6), structure 550 (FIG. 7), method 420 (FIG. 8), and / or method 440 (FIG. 9), etc. Additionally, audio I / O 1024 may be coupled to second bus 1020, and battery 1010 may provide power to computing system 1000.

[0107] It should be noted that other embodiments are contemplated. For example, rather than the point-to-point architecture of Figure 12, the system may implement a multi-drop bus or another such communication topology. Also, the elements of Figure 12 may alternatively be divided using more or fewer integrated chips than those shown in Figure 12.

[0108] Additional Notes and Examples

[0109] Example 1 includes a computing system comprising a system-on-chip having a plurality of processing engines and a memory having a set of executable program instructions that, when executed by the system-on-chip, cause the computing system to: determine, using a first processing engine of the plurality of processing engines, a first partial similarity measure based on a first portion of a query vector and a first portion of a first candidate vector; determine, using a second processing engine of the plurality of processing engines, a total similarity measure based on the query vector and a second candidate vector; and determine, using the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure.

[0110] Example 2 includes the computing system of Example 1, wherein the instructions, when executed, further cause the computing system to, in response to the first partial similarity measure being less than the total similarity measure, use the first processing engine to compare the second portion of the query vector to the second portion of the first candidate vector, wherein the first partial similarity measure is a partial distance and the total similarity measure is a total distance.

[0111] Example 3 includes the computing system of Example 1, wherein the instructions, when executed, further cause the computing system to: use the plurality of processing engines to retrieve a distinct candidate vector; use the plurality of processing engines to determine a plurality of partial similarity measures between a first portion of the query vector and a first portion of the distinct candidate vector; and use the plurality of processing engines to determine, based on the plurality of partial similarity measures and the total similarity measure, whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the distinct candidate vector.

[0112] Example 4 includes the computing system of Example 3, wherein the system-on-chip has a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database.

[0113] Example 5 includes the computing system of Example 1, wherein the instructions, when executed, further cause the computing system to: determine, using the first processing engine, to bypass a partial similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, where an index for the query vector is a value at which the first partial similarity measure is determined; and, in response to the partial similarity calculation being bypassed, increment, using the first processing engine, the value of the index; and determine, using the first processing engine, whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure to be determined based on feature values ​​of a third candidate vector and feature values ​​of the query vector, where both the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index.

[0114] Example 6 includes the computing system of any one of Examples 1-5, wherein the instructions, when executed, further cause the computing system to store the total similarity measure and a plurality of similarity measures in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measures being determined based on different candidate vectors and the query vector, and the total similarity measure is greater than each of the plurality of similarity measures.

[0115] Example 7 includes the computing system of Example 1, wherein the instructions, when executed, further cause the computing system to: store a plurality of candidate vectors in a plurality of ping-pong buffers; determine, with the plurality of processing engines, a plurality of partial similarity measures based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors; and determine, with the plurality of processing engines, based on the first subset of the plurality of partial similarity measures and a first total similarity measure, that similarity calculations associated with a first subset of the plurality of candidate vectors will be bypassed.

[0116] Example 8 includes the computing system of Example 7, wherein the instructions, when executed, further cause the computing system to: determine, using the group of the plurality of processing engines, that a second subset of the plurality of candidate vectors is to be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; determine, using the group of the plurality of processing engines, a second total similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; and store each respective total similarity measure of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure, each of the different heap memories being dedicated to one of the plurality of query vectors.

[0117] Example 9 includes a semiconductor device comprising one or more substrates and logic coupled to the one or more substrates, the logic implemented in one or more of configurable logic or fixed function logic hardware, the logic coupled to the one or more substrates: determining, with a first processing engine of a plurality of processing engines, a first partial similarity measure based on a first portion of a query vector and a first portion of a first candidate vector; determining, with a second processing engine of the plurality of processing engines, a total similarity measure based on the query vector and a second candidate vector; and determining, with the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure.

[0118] Example 10 includes the apparatus of Example 9, wherein the logic coupled to the one or more substrates, in response to the first partial similarity measure being less than the total similarity measure, uses the first processing engine to compare the second portion of the query vector to the second portion of the first candidate vector, wherein the first partial similarity measure is a partial distance and the total similarity measure is a total distance.

[0119] Example 11 includes the apparatus of Example 9, wherein the logic coupled to the one or more substrates uses the plurality of processing engines to retrieve distinct candidate vectors; uses the plurality of processing engines to determine a plurality of partial similarity measures between a first portion of the query vector and a first portion of the distinct candidate vector; and uses the plurality of processing engines to determine, based on the plurality of partial similarity measures and the total similarity measure, whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the distinct candidate vector.

[0120] Example 12 includes the apparatus of Example 11, wherein the logic coupled to the one or more substrates accesses a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database.

[0121] Example 13 includes the apparatus of Example 9, wherein the logic coupled to the one or more substrates: determines, with the first processing engine, to bypass a similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, wherein an index for the query vector is a value at which the first partial similarity measure is determined; and, in response to the similarity calculation for the first candidate vector being bypassed, increments, with the first processing engine, the value of the index; and determines, with the first processing engine, whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure that is determined based on feature values ​​of a third candidate vector and feature values ​​of the query vector, wherein both the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index.

[0122] Example 14 includes the apparatus of any one of Examples 9-13, wherein the logic coupled to the one or more substrates stores the total similarity measure and a plurality of similarity measures in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measures being determined based on different candidate vectors and the query vector, and the total similarity measure is greater than each of the plurality of similarity measures.

[0123] Example 15 includes the apparatus of Example 9, wherein the logic coupled to the one or more substrates stores a plurality of candidate vectors in a plurality of ping-pong buffers; determines, with the plurality of processing engines, a plurality of partial similarity measures based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors; and determines, with the plurality of processing engines, based on the first subset of the plurality of partial similarity measures and a first total similarity measure, that a similarity calculation associated with a first subset of the plurality of candidate vectors is to be bypassed.

[0124] Example 16 includes the apparatus of Example 15, wherein the logic coupled to the one or more substrates determines, using a group of the plurality of processing engines, that a second subset of the plurality of candidate vectors is to be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; determines, using the group of the plurality of processing engines, a second total similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; and stores each respective total similarity measure of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure, each of the different heap memories dedicated to one of the plurality of query vectors.

[0125] Example 17 includes the device of example 9, wherein the logic coupled to the one or more substrates includes a transistor channel region positioned within the one or more substrates.

[0126] Example 18 includes at least one computer-readable storage medium comprising a set of executable program instructions that, when executed by a computing system, cause the computing system to: determine, with a first processing engine of a plurality of processing engines, a first partial similarity measure based on a first portion of a query vector and a first portion of a first candidate vector; determine, with a second processing engine of the plurality of processing engines, a total similarity measure based on the query vector and a second candidate vector; and determine, with the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure.

[0127] Example 19 includes at least one computer-readable storage medium of Example 18, wherein the instructions, when executed, further cause the computing system to, in response to the first partial similarity measure being less than the total similarity measure, use the first processing engine to compare the second portion of the query vector to the second portion of the first candidate vector, wherein the first partial similarity measure is a partial distance and the total similarity measure is a total distance.

[0128] Example 20 includes at least one computer-readable storage medium as recited in Example 18, wherein the instructions, when executed, further cause the computing system to: use the plurality of processing engines to retrieve a distinct candidate vector; use the plurality of processing engines to determine a plurality of partial similarity measures between a first portion of the query vector and a first portion of the distinct candidate vector; and use the plurality of processing engines to determine, based on the plurality of partial similarity measures and the total similarity measure, whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the distinct candidate vector.

[0129] Example 21 includes at least one computer-readable storage medium as described in Example 20, wherein the instructions, when executed, further cause the computing system to access a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database.

[0130] Example 22 includes at least one computer-readable storage medium of Example 18, wherein the instructions, when executed, further cause the computing system to: determine, using the first processing engine, to bypass a partial similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, wherein an index for the query vector is a value at which the first partial similarity measure is determined; and, in response to the partial similarity calculation being bypassed, increment, using the first processing engine, the value of the index; and determine, using the first processing engine, whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure to be determined based on feature values ​​of a third candidate vector and feature values ​​of the query vector, wherein both the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index.

[0131] Example 23 includes at least one computer-readable storage medium of any one of Examples 18-22, wherein the instructions, when executed, further cause the computing system to store the total similarity measure and a plurality of similarity measures in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measures being determined based on different candidate vectors and the query vector, and the total similarity measure being greater than each of the plurality of similarity measures.

[0132] Example 24 includes at least one computer-readable storage medium of Example 18, wherein the instructions, when executed, further cause the computing system to: store a plurality of candidate vectors in a plurality of ping-pong buffers; determine, with the plurality of processing engines, a plurality of partial similarity measures based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors; and determine, with the plurality of processing engines, that a similarity calculation associated with a first subset of the plurality of candidate vectors is to be bypassed based on the first subset of the plurality of partial similarity measures and a first total similarity measure.

[0133] Example 25 includes the at least one computer-readable storage medium of Example 24, wherein the instructions, when executed, further cause the computing system to: determine, with the group of the plurality of processing engines, that a second subset of the plurality of candidate vectors is to be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; determine, with the group of the plurality of processing engines, a second total similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; and store each respective total similarity measure of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure, each of the different heap memories being dedicated to one of the plurality of query vectors.

[0134] Example 26 includes a semiconductor device comprising: means for determining, using a first processing engine of a plurality of processing engines, a first partial similarity measure based on a first portion of a query vector and a first portion of a first candidate vector; means for determining, using a second processing engine of the plurality of processing engines, a total similarity measure based on the query vector and a second candidate vector; and means for determining, using the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure.

[0135] Example 27 includes the apparatus of Example 26, further comprising means for, in response to the first partial similarity measure being less than the total similarity measure, using the first processing engine, comparing the second portion of the query vector to the second portion of the first candidate vector, wherein the first partial similarity measure is a partial distance and the total similarity measure is a total distance.

[0136] Example 28 includes the apparatus of Example 26, further comprising: means for retrieving a distinct candidate vector using the plurality of processing engines; means for determining, using the plurality of processing engines, a plurality of partial similarity measures between a first portion of the query vector and a first portion of the distinct candidate vector; and means for determining, using the plurality of processing engines, whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the distinct candidate vector based on the plurality of partial similarity measures and the total similarity measure.

[0137] Example 29 includes the apparatus of Example 28, further comprising means for accessing a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database.

[0138] Example 30 includes the apparatus of Example 26, further comprising: means for determining, using the first processing engine, to bypass a similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, where an index for the query vector is a value at which the first partial similarity measure is determined; means for incrementing, using the first processing engine, the value of the index in response to the similarity calculation for the first candidate vector being bypassed; and means for determining, using the first processing engine, whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure that is determined based on feature values ​​of a third candidate vector and feature values ​​of the query vector, where both the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index.

[0139] Example 31 includes the apparatus of any one of Examples 26-30, further comprising means for storing the total similarity measure and a plurality of similarity measures in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measures being determined based on different candidate vectors and the query vector, and the total similarity measure being greater than each of the plurality of similarity measures.

[0140] Example 32 includes the apparatus of Example 26, further comprising: means for storing a plurality of candidate vectors in a plurality of ping-pong buffers; means for determining, with the plurality of processing engines, a plurality of partial similarity measures based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors; and means for determining, with the plurality of processing engines, based on a first subset of the plurality of partial similarity measures and a first total similarity measure, that a similarity calculation associated with a first subset of the plurality of candidate vectors is to be bypassed.

[0141] Example 33 includes the apparatus of Example 32, further comprising: means for determining, using the group of the plurality of processing engines, that a second subset of the plurality of candidate vectors will be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; means for determining, using the group of the plurality of processing engines, a second total similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; and means for storing each respective total similarity measure of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure, each of the different heap memories being dedicated to one of the plurality of query vectors.

[0142] In this manner, the techniques described herein may provide improved matching and query analysis that may efficiently retrieve results. Furthermore, queries may be batch processes to facilitate low-latency analysis. Also, the embodiments described herein may reduce memory footprint and latency, as well as processing power.

[0143] The embodiments are applicable for use with all types of semiconductor integrated circuit ("IC") chips. Examples of these IC chips include, but are not limited to, processors, controllers, chipset components, programmable logic arrays (PLAs), memory chips, network chips, systems-on-chips (SoCs), SSD / NAND controller ASICs, etc. Additionally, in some of the figures, signal conductors are represented by lines. Some may be different to indicate more component signal paths, may have numbered labels to indicate multiple component signal paths, and / or may have arrows on one or more ends to indicate the primary direction of information flow. However, this should not be construed as limiting. Rather, such additional detail may be used in connection with one or more exemplary embodiments to facilitate easier understanding of the circuit. Any signal line shown, whether or not it has additional information, may actually include one or more signals that can travel in multiple directions and may be implemented with any suitable type of signaling, such as, for example, digital or analog lines implemented in differential pairs, fiber optic lines, and / or single-ended lines.

[0144] While example sizes / models / values / ranges may be given, the embodiments are not limited thereto. It is expected that, as manufacturing techniques (e.g., photolithography) mature over time, smaller devices may be manufactured. Additionally, for simplicity of illustration and discussion, and so as not to obscure certain aspects of the embodiments, known power / ground connections to IC chips and other components may or may not be shown in the figures. Furthermore, to avoid obscuring the embodiments, and given that details regarding the implementation of such block diagram configurations are highly dependent on the platform on which the embodiments are to be implemented, i.e., such details are well within the knowledge of those skilled in the art, configurations may be shown in block diagram form. Where specific details (e.g., circuits) are described to explain example embodiments, it will be apparent to one skilled in the art that the embodiments can be practiced without these specific details or with variations on these specific details. Therefore, the description should be considered as illustrative and not limiting.

[0145] The term "coupled" may be used herein to refer to any type of direct or indirect relationship between components of interest and may apply to electrical, mechanical, fluid, optical, electromagnetic, electromechanical, or other connections. Additionally, terms such as "first," "second," etc. may be used herein solely for ease of discussion and do not have any particular temporal or chronological meaning unless otherwise indicated.

[0146] As used in this application and the claims, a list of items joined by the term "one or more of" may mean any combination of the listed terms. For example, the phrase "one or more of A, B, or C" may mean A, B, C, A and B, A and C, B and C, or A, B, and C.

[0147] Those skilled in the art will appreciate from the foregoing description that the broad techniques of the embodiments can be implemented in a variety of forms. Thus, while the embodiments have been described with reference to specific examples thereof, the true scope of the embodiments should not be limited thereto, as other modifications will become apparent to those skilled in the art upon review of the drawings, the specification, and the following claims. [Other possible items] [Item 1] a system-on-chip having a plurality of processing engines; a memory having a set of executable program instructions; wherein the instructions, when executed by the system-on-chip, cause the computing system to: determining, with a first processing engine of the plurality of processing engines, a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector; determining, using a second processing engine of the plurality of processing engines, an overall similarity measure based on the query vector and a second candidate vector; determining, with the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure; A computing system that performs the following: [Item 2] The instructions, when executed, cause the computing system to: 2. The computing system of claim 1, further causing the first processing engine to compare, in response to the first partial similarity measure being less than the total similarity measure, the second portion of the query vector with the second portion of the first candidate vector, wherein the first partial similarity measure is a partial distance and the total similarity measure is a total distance. [Item 3] The instructions, when executed, cause the computing system to: retrieving different candidate vectors using the plurality of processing engines; determining, with the plurality of processing engines, a plurality of partial similarity measures between a first portion of the query vector and a first portion of the different candidate vectors; using the plurality of processing engines to determine whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the different candidate vector based on the plurality of partial similarity measures and the total similarity measure; Item 1. The computing system of item 1, further comprising: [Item 4] 4. The computing system of claim 3, wherein the system-on-chip has a plurality of memory storage areas each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database. [Item 5] The instructions, when executed, cause the computing system to: determining, using the first processing engine, to bypass a partial similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, wherein an index for the query vector is a value at which the first partial similarity measure is determined; incrementing, with the first processing engine, the value of the index in response to the partial similarity calculation being bypassed; using the first processing engine to determine whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure determined based on feature values ​​of a third candidate vector and feature values ​​of the query vector; 2. The computing system of claim 1, further comprising: a step of: associating the feature value of the third candidate vector and the feature value of the query vector with the incremented value of the index; [Item 6] The instructions, when executed, cause the computing system to: 2. The computing system of claim 1, further comprising: storing the total similarity measure and a plurality of similarity measures in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measures being determined based on different candidate vectors and the query vector, and the total similarity measure being greater than each of the plurality of similarity measures. [Item 7] The instructions, when executed, cause the computing system to: storing a plurality of candidate vectors in a plurality of ping-pong buffers; determining, with the plurality of processing engines, a plurality of partial similarity measures based on first portions of a plurality of query vectors and first portions of the plurality of candidate vectors; determining, with the plurality of processing engines, based on the first subset of the plurality of partial similarity measures and a first total similarity measure, that similarity calculations associated with a first subset of the plurality of candidate vectors will be bypassed; Item 1. The computing system of item 1, further comprising: [Item 8] The instructions, when executed, cause the computing system to: determining, using the group of processing engines, a second subset of the plurality of candidate vectors to be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; determining, using the group of the plurality of processing engines, a second overall similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; storing each respective one of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure; 8. The computing system of claim 7, further comprising: a processor configured to: [Item 9] one or more substrates; logic coupled to said one or more substrates; 1. A semiconductor device comprising: the logic implemented in one or more of configurable logic or fixed function logic hardware; and the logic coupled to the one or more substrates, determining, with a first processing engine of the plurality of processing engines, a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector; determining, using a second processing engine of the plurality of processing engines, an overall similarity measure based on the query vector and a second candidate vector; determining, with the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure; A semiconductor device that performs the above. [Item 10] The logic coupled to the one or more substrates includes: Item 10. The apparatus of item 9, wherein, in response to the first partial similarity measure being less than the total similarity measure, using the first processing engine, compares the second portion of the query vector to the second portion of the first candidate vector, wherein the first partial similarity measure is a partial distance and the total similarity measure is a total distance. [Item 11] The logic coupled to the one or more substrates includes: retrieving different candidate vectors using the plurality of processing engines; determining, with the plurality of processing engines, a plurality of partial similarity measures between a first portion of the query vector and a first portion of the different candidate vectors; using the plurality of processing engines to determine whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the different candidate vector based on the plurality of partial similarity measures and the total similarity measure; Item 10. The apparatus according to Item 9, [Item 12] Item 12. The apparatus of item 11, wherein the logic coupled to the one or more substrates accesses a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database. [Item 13] The logic coupled to the one or more substrates includes: determining, using the first processing engine, to bypass a similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, wherein an index for the query vector is a value at which the first partial similarity measure is determined; incrementing, with the first processing engine, the value of the index in response to the similarity calculation for the first candidate vector being bypassed; using the first processing engine to determine whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure determined based on feature values ​​of a third candidate vector and feature values ​​of the query vector; and the feature value of the third candidate vector and the feature value of the query vector are both associated with the incremented value of the index. [Item 14] The logic coupled to the one or more substrates includes: 10. The apparatus of claim 9, wherein the total similarity measure and the plurality of similarity measures are stored in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measures are determined based on different candidate vectors and the query vector, and the total similarity measure is greater than each of the plurality of similarity measures. [Item 15] The logic coupled to the one or more substrates includes: storing a plurality of candidate vectors in a plurality of ping-pong buffers; determining, with the plurality of processing engines, a plurality of partial similarity measures based on first portions of a plurality of query vectors and first portions of the plurality of candidate vectors; determining, with the plurality of processing engines, based on the first subset of the plurality of partial similarity measures and a first total similarity measure, that similarity calculations associated with a first subset of the plurality of candidate vectors will be bypassed; Item 10. The apparatus according to Item 9, [Item 16] The logic coupled to the one or more substrates includes: determining, using the group of processing engines, a second subset of the plurality of candidate vectors to be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; determining, using the group of the plurality of processing engines, a second overall similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; storing each respective one of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure; 16. The apparatus of claim 15, wherein each of the different heap memories is dedicated to one of the plurality of query vectors. [Item 17] 10. The device of claim 9, wherein the logic coupled to the one or more substrates includes a transistor channel region positioned within the one or more substrates. [Item 18] At least one computer-readable storage medium comprising a set of executable program instructions that, when executed by a computing system, cause the computing system to: determining, with a first processing engine of the plurality of processing engines, a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector; determining, using a second processing engine of the plurality of processing engines, an overall similarity measure based on the query vector and a second candidate vector; determining, with the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure; At least one computer-readable storage medium that causes the [Item 19] The instructions, when executed, cause the computing system to: Item 19. The at least one computer-readable storage medium of item 18, further causing the first processing engine to compare, in response to the first partial similarity measure being less than the total similarity measure, the second portion of the query vector with the second portion of the first candidate vector, wherein the first partial similarity measure is a partial distance and the total similarity measure is a total distance. [Item 20] The instructions, when executed, cause the computing system to: retrieving different candidate vectors using the plurality of processing engines; determining, with the plurality of processing engines, a plurality of partial similarity measures between a first portion of the query vector and a first portion of the different candidate vectors; using the plurality of processing engines to determine whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the different candidate vector based on the plurality of partial similarity measures and the total similarity measure; 20. At least one computer-readable storage medium according to item 18, further comprising: [Item 21] The instructions, when executed, cause the computing system to: 22. The at least one computer-readable storage medium of claim 20, further comprising: accessing a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database. The instructions, when executed, cause the computing system to: determining, using the first processing engine, to bypass a partial similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, wherein an index for the query vector is a value at which the first partial similarity measure is determined; incrementing, with the first processing engine, the value of the index in response to the partial similarity calculation being bypassed; using the first processing engine to determine whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure determined based on feature values ​​of a third candidate vector and feature values ​​of the query vector; 20. The at least one computer-readable storage medium of claim 18, further comprising: a step of: associating the feature value of the third candidate vector and the feature value of the query vector with the incremented value of the index; [Item 23] The instructions, when executed, cause the computing system to: Item 19. The at least one computer-readable storage medium of item 18, further comprising storing the total similarity measure and a plurality of similarity measures in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measures being determined based on different candidate vectors and the query vector, and the total similarity measure being greater than each of the plurality of similarity measures. [Item 24] The instructions, when executed, cause the computing system to: storing a plurality of candidate vectors in a plurality of ping-pong buffers; determining, with the plurality of processing engines, a plurality of partial similarity measures based on first portions of a plurality of query vectors and first portions of the plurality of candidate vectors; determining, with the plurality of processing engines, that similarity calculations associated with a first subset of the plurality of candidate vectors will be bypassed based on a first subset of the plurality of partial similarity measures and a first total similarity measure; 20. At least one computer-readable storage medium according to item 18, further comprising: [Item 25] The instructions, when executed, cause the computing system to: determining, using the group of processing engines, a second subset of the plurality of candidate vectors to be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; determining, using the group of the plurality of processing engines, a second overall similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; storing each respective one of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure; 25. The at least one computer-readable storage medium of claim 24, further comprising:

Claims

1. a system-on-chip having a plurality of processing engines; a memory having a set of executable program instructions; wherein the executable program instructions, when executed by the system-on-chip, provide the computing system with: determining, with a first processing engine of the plurality of processing engines, a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector; determining, using a second processing engine of the plurality of processing engines, an overall similarity measure based on the query vector and a second candidate vector; determining, with the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure; A computing system that performs the following:

2. The executable program instructions, when executed, cause the computing system to:

2. The computing system of claim 1, further causing, in response to the first partial similarity measure being less than the total similarity measure, the first processing engine to compare the second portion of the query vector with the second portion of the first candidate vector, wherein the first partial similarity measure is a partial distance and the total similarity measure is a total distance.

3. The executable program instructions, when executed, cause the computing system to: retrieving different candidate vectors using the plurality of processing engines; determining, with the plurality of processing engines, a plurality of partial similarity measures between a first portion of the query vector and a first portion of the different candidate vectors; using the plurality of processing engines to determine whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the different candidate vector based on the plurality of partial similarity measures and the total similarity measure; The computing system of claim 1 or 2, further comprising:

4. 4. The computing system of claim 3, wherein the system-on-chip has a plurality of memory storage areas each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database.

5. The executable program instructions, when executed, cause the computing system to: determining, using the first processing engine, to bypass a partial similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, wherein an index for the query vector is a value at which the first partial similarity measure is determined; incrementing, with the first processing engine, the value of the index in response to the partial similarity calculation being bypassed; using the first processing engine to determine whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure determined based on feature values ​​of the third candidate vector and feature values ​​of the query vector; 2. The computing system of claim 1, further comprising: a step of: associating the feature value of the third candidate vector and the feature value of the query vector with the incremented value of the index.

6. The executable program instructions, when executed, cause the computing system to:

6. The computing system of claim 1, further comprising: storing the total similarity measure and a plurality of similarity measures in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measures being determined based on different candidate vectors and the query vector, and the total similarity measure being greater than each of the plurality of similarity measures.

7. The executable program instructions, when executed, cause the computing system to: storing a plurality of candidate vectors in a plurality of ping-pong buffers; determining, with the plurality of processing engines, a plurality of partial similarity measures based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors; determining, with the plurality of processing engines, based on the first subset of the plurality of partial similarity measures and a first total similarity measure, that similarity calculations associated with a first subset of the plurality of candidate vectors will be bypassed; The computing system of claim 1 further comprising:

8. The executable program instructions, when executed, cause the computing system to: determining, using the group of processing engines, a second subset of the plurality of candidate vectors to be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; determining, using the group of the plurality of processing engines, a second overall similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; storing each respective one of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure; 8. The computing system of claim 7, further comprising: a) dedicating a plurality of query vectors to each of the plurality of heap memories;

9. one or more substrates; logic coupled to said one or more substrates; 1. A semiconductor device comprising: the logic implemented in one or more of configurable logic or fixed function logic hardware; and the logic coupled to the one or more substrates, determining, with a first processing engine of the plurality of processing engines, a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector; determining, using a second processing engine of the plurality of processing engines, an overall similarity measure based on the query vector and a second candidate vector; determining, with the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure; A semiconductor device that performs the above.

10. The logic coupled to the one or more substrates includes:

10. The semiconductor device of claim 9, wherein the first processing engine is used to compare the second portion of the query vector with the second portion of the first candidate vector in response to the first partial similarity measure being less than the total similarity measure, the first partial similarity measure being a partial distance and the total similarity measure being a total distance.

11. The logic coupled to the one or more substrates includes: retrieving different candidate vectors using the plurality of processing engines; determining, with the plurality of processing engines, a plurality of partial similarity measures between a first portion of the query vector and a first portion of the different candidate vectors; using the plurality of processing engines to determine whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the different candidate vector based on the plurality of partial similarity measures and the total similarity measure; The semiconductor device according to claim 9 or 10, wherein the steps of:

12. 12. The semiconductor device of claim 11, wherein the logic coupled to the one or more substrates accesses a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database.

13. The logic coupled to the one or more substrates includes: determining, using the first processing engine, to bypass a similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, wherein an index for the query vector is a value at which the first partial similarity measure is determined; incrementing, with the first processing engine, the value of the index in response to the similarity calculation for the first candidate vector being bypassed; using the first processing engine to determine whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure determined based on feature values ​​of a third candidate vector and feature values ​​of the query vector; and both the feature value of the third candidate vector and the feature value of the query vector are associated with the incremented value of the index.

14. The logic coupled to the one or more substrates includes:

14. The semiconductor device according to claim 9, wherein the total similarity measure and the plurality of similarity measure values ​​are stored in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measure values ​​are determined based on different candidate vectors and the query vector, and the total similarity measure value is greater than each of the plurality of similarity measure values.

15. The logic coupled to the one or more substrates includes: storing a plurality of candidate vectors in a plurality of ping-pong buffers; determining, with the plurality of processing engines, a plurality of partial similarity measures based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors; determining, with the plurality of processing engines, based on the first subset of the plurality of partial similarity measures and a first total similarity measure, that similarity calculations associated with a first subset of the plurality of candidate vectors will be bypassed; The semiconductor device according to claim 9 , wherein the semiconductor device performs the steps of:

16. The logic coupled to the one or more substrates includes: determining, using the group of processing engines, a second subset of the plurality of candidate vectors to be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; determining, using the group of the plurality of processing engines, a second overall similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; storing each respective one of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure; 16. The semiconductor device of claim 15, wherein each of the different heap memories is dedicated to one of the plurality of query vectors.

17. 10. The semiconductor device of claim 9, wherein the logic coupled to the one or more substrates includes a transistor channel region positioned within the one or more substrates.

18. A computer program comprising a set of executable instructions that, when executed by a computing system, cause the computing system to: determining, with a first processing engine of the plurality of processing engines, a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector; determining, using a second processing engine of the plurality of processing engines, an overall similarity measure based on the query vector and a second candidate vector; determining, with the first processing engine, whether to compare a second portion of the query vector to a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure; A computer program that performs the following:

19. The instructions, when executed, cause the computing system to:

20. The computer program product of claim 18, further causing, in response to the first partial similarity measure being less than the total similarity measure, using the first processing engine to compare the second portion of the query vector to the second portion of the first candidate vector, wherein the first partial similarity measure is a partial distance and the total similarity measure is a total distance.

20. The instructions, when executed, cause the computing system to: retrieving different candidate vectors using the plurality of processing engines; determining, with the plurality of processing engines, a plurality of partial similarity measures between a first portion of the query vector and a first portion of the different candidate vectors; using the plurality of processing engines to determine whether to bypass a partial similarity calculation between a second portion of the query vector and a second portion of the different candidate vector based on the plurality of partial similarity measures and the total similarity measure; 20. The computer program of claim 18 or 19, further comprising:

21. The instructions, when executed, cause the computing system to:

21. The computer program product of claim 20, further comprising accessing a plurality of memory storage areas each dedicated to one of the plurality of processing engines, the plurality of memory storage areas storing the different candidate vectors, the different candidate vectors representing a vector candidate database.

22. The instructions, when executed, cause the computing system to: determining, using the first processing engine, to bypass a partial similarity calculation for the first candidate vector based on the first partial similarity measure and the total similarity measure, wherein an index for the query vector is a value at which the first partial similarity measure is determined; incrementing, with the first processing engine, the value of the index in response to the partial similarity calculation being bypassed; using the first processing engine to determine whether to bypass a similarity calculation for the third candidate vector based on a partial similarity measure determined based on feature values ​​of a third candidate vector and feature values ​​of the query vector; wherein the feature value of the third candidate vector and the feature value of the query vector are both associated with the incremented value of the index.

23. The instructions, when executed, cause the computing system to:

23. The computer program of claim 18, further comprising storing the total similarity measure and a plurality of similarity measures in a max-heap binary tree or a min-heap binary tree, the plurality of similarity measures being determined based on different candidate vectors and the query vector, and the total similarity measure being greater than each of the plurality of similarity measures.

24. The instructions, when executed, cause the computing system to: storing a plurality of candidate vectors in a plurality of ping-pong buffers; determining, with the plurality of processing engines, a plurality of partial similarity measures based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors; determining, with the plurality of processing engines, that similarity calculations associated with a first subset of the plurality of candidate vectors will be bypassed based on a first subset of the plurality of partial similarity measures and a first total similarity measure; 20. The computer program of claim 18, further comprising:

25. The instructions, when executed, cause the computing system to: determining, using the group of processing engines, a second subset of the plurality of candidate vectors to be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; determining, using the group of the plurality of processing engines, a second overall similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; storing each respective one of the second total similarity measures in a different heap memory based on an identity of a query vector of the plurality of query vectors associated with the respective total similarity measure; 25. The computer program product of claim 24, further comprising: each of the different heap memories dedicated to one of the plurality of query vectors.

26. At least one computer readable storage medium storing a computer program according to any one of claims 18 to 25.

Images