Parallel pruning and batch sort for similarity search accelerators

Abstract

The present invention provides a computing system, semiconductor device, program, and storage medium that perform parallel pruning and similarity calculations for candidate vectors and query vectors using a similarity processing engine. [Solution] The method involves using a first processing engine among multiple processing engines to determine a first partial similarity measurement based on a first portion of the query vector and a first portion of the first candidate vector; using a second processing engine among multiple processing engines to determine a total similarity measurement based on the query vector and a second candidate vector; and using the first processing engine to determine whether or not to compare a second portion of the query vector with a second portion of the first candidate vector based on the first partial similarity measurement and the total similarity measurement.

Description

Technical Field

[0001] Embodiments generally relate to a processing architecture that performs parallel pruning and similarity calculation for candidate vectors and query vectors using a similarity processing engine. Embodiments include shared heap hardware that sorts the results from parallel similarity processing engines.

Background Art

[0002] Content-based similarity search (e.g., similarity search) can be performed by machine learning (ML) and / or artificial intelligence (AI) applications (e.g., recommendation engines, visual search engines, drug discovery, etc.). For example, a database can include a large number (e.g., billions) of high-dimensional candidate vectors. Query vectors q of the same dimension, format, and size (e.g., 512 bytes) are collated against the database (e.g., based on some similarity function such as Euclidean similarity measurement), and database vectors similar to and / or closest to the query vector q can be identified. For example, a content-based image retrieval (CBIR) system can use a query image that is decomposed into query vectors and then collated against candidate vectors representing similar images to identify those similar images in the database. The feature extraction stage can involve a deep learning model. Moreover, in modern applications, these vectors can represent a wide range of categories such as image content, text, web search, protein sequences, faces, sounds, or bioinformatics data, which are extracted and summarized by deep learning systems.

Brief Description of the Drawings

[0003] Various advantages of the embodiments will become apparent to those skilled in the art by reading the following specification and the appended claims, and by referring to the following drawings.

[0004] [Figure 1] It is a diagram of an example of a near-memory similarity search architecture according to an embodiment.

[0005] [Figure 2A] This figure shows an example of a far-memory similarity search and matching architecture according to one embodiment. [Figure 2B] This figure shows an example of a far-memory similarity search and matching architecture according to one embodiment.

[0006] [Figure 3] This is a flowchart of an example of a similarity search method using pruning according to one embodiment.

[0007] [Figure 4] This is a timing diagram of an example of a similarity search using a processing engine according to one embodiment.

[0008] [Figure 5] This is a flowchart of an example of a query streaming method according to one embodiment.

[0009] [Figure 6] This is a flowchart of an example of a similarity calculation method according to one embodiment.

[0010] [Figure 7] This figure shows an example of a heap memory structure according to one embodiment.

[0011] [Figure 8] This is a flowchart illustrating an example of a method for inserting a new node entry into an unfilled heap memory structure according to one embodiment.

[0012] [Figure 9] This is a flowchart illustrating an example of a method for inserting a new node entry into a full heap memory structure according to one embodiment.

[0013] [Figure 10]It is a block diagram of an example of a similarity search improvement computing system according to an embodiment.

[0014] [Figure 11] It is an illustration of an example of a semiconductor device according to an embodiment.

[0015] [Figure 12] It is a block diagram of an example of a processor according to an embodiment.

[0016] [Figure 13] It is a block diagram of an example of a multiprocessor-based computing system according to an embodiment.

Embodiments of the Invention

[0017] Referring now to FIG. 1, a similarity search architecture 100 (e.g., a system-on-chip) implements an improved similarity and early pruning search process that runs with reduced latency, a narrower bandwidth, and reduced computing resources. Specifically, the similarity search architecture 100 uses an array of similarity processing engines (PEs) 110 (e.g., configurable logic, fixed-function logic hardware, processing elements, execution units, etc.) to perform parallel similarity calculations to identify the closest similarity match between the first through N query vectors 106a - 1(n) and the candidate vectors V000 - Vn03. As will be described in more detail, the similarity PEs 110 perform early pruning (e.g., bypassing) to discard similarity processing at an early stage, reducing computing resources and processing power. Further, the similarity PEs 110 operate using a near-memory architecture that includes a memory area 108 to reduce bandwidth and communication latency. Moreover, the architecture 100 may operate over a batch of query vectors 106, while the candidate vectors V000 - Vn03 are stored in the memory area 108, avoiding the high-latency data retrieval of the candidate vectors V000 - Vn03 from long-term storage.

[0018] In detail, the similarity PE110 may determine various similarity measures (e.g., Manhattan distance, Euclidean distance, Minkowski and Hamming distance, cosine and / or dot product similarity). Similarity may be determined by distance metrics such as Manhattan distance, Euclidean distance, Minkowski and Hamming distance, such that lower distances correspond to higher similarity. In some examples, PE110 may prune (e.g., bypass) future similarity calculations (e.g., similarity measurements) between each candidate vector of candidate vectors V000~Vn03 and each query vector of query vector 106 if the partially calculated distance between them is higher than a threshold (e.g., the longest distance and / or lowest similarity score, such as determined from the top k distances). Such pruning is performed at an early stage before determining all similarity measurements between the vector features of each candidate vector and the vector features of each query vector, in which case it is determined whether to ignore each candidate vector. By doing so, computational resources and latency can be reduced without compromising accuracy. Each of the memory areas 108 contains four candidate vectors V000 to Vn03, but such numbers are illustrative and it will be understood that embodiments described herein are not limited thereto. In fact, each of the memory areas 108 may store any number of candidate vectors (e.g., M) and operate in the same manner as described herein.

[0019] Furthermore, some embodiments include first through Nth memory areas 108a through 108n (e.g., static random access memory banks) each storing a subset of candidate vectors V000 through Vn03 and each dedicated to one of the degrees of similarity PE110. The plurality of degrees of similarity PE110 may be executed efficiently in parallel and based on different candidate vectors of the candidate vectors V000 through Vn03 retrieved from the first through Nth memory areas 108a through 108n. By doing this, the latency can be further reduced. This is because each of the first through Nth degrees of similarity PE110a through 110n can have less idling and waiting due to blocking and waiting for the calculation of other degrees of similarity PE110a through 110n. For example, each of the first through Nth degrees of similarity PE110a through 110n may operate independently of the other first through Nth degrees of similarity PE110a through 110n to perform a similarity search on a subset of the candidate vectors V000 through Vn03.

[0020] In addition, the first through Nth degrees of similarity PE110a through 110n may process a batch of the first through Nth query vectors 106a through 106n in series. By doing this, memory fetches and power consumption can be reduced. This is because the candidate vectors V000 through Vn03 may remain within the first through Nth memory areas 108a through 108n throughout the processing of the first through Nth query vectors 106a through 106n. For example, the degree of similarity PE110 may access a continuous query stream of the query vectors 106 to perform a similarity search.

[0021] As described herein, features may be useful pieces of information extracted from data. The size of a feature may be in bytes (e.g., 1 byte) and may also be referred to as a dimension. High-dimensional inputs such as images may be reduced to a reduced number of dimensions or features. A single vector (query vector and / or candidate vector, etc.) contains multiple features to form a feature vector. The size of a feature vector is determined by the number of features and the size of each feature (e.g., 1 byte, INT8 format size, INT16 format size, INT32 format size, FP32 format size, BF16 format size). A query (or query vector) may be a feature vector extracted from a running application or user search. For example, if a face detection application is used, the query vector will be a feature vector extracted from the query face, and the query vector may be compared against candidate vectors (each representing a candidate face) to identify matching faces.

[0022] In one embodiment, the similarity PE110 determines a distance representing the similarity between two vectors (e.g., it quantifies the similarity between two vectors). That is, the similarity PE110 calculates the distance or similarity between each query vector of query vector 106 and each candidate vector of candidate vectors V000~Vn03 at a given time point. For example, a single similarity PE among several similarity PE110s may be executed over a certain number (e.g., 512) clock cycles to compare each query vector with each candidate vector, assuming a corresponding feature size (e.g., 512 bytes if the data path is 1 byte wide).

[0023] As described, the similarity PE110 determines distance as a similarity measure. Embodiments as described herein may determine other similarity measures and operate similarly to those described herein for distance. Thus, architecture 100 utilizes the mathematical properties of distance and similarity algorithms to cyclically calculate similarity across features in the query vector 106 and candidate vectors V000~Vn03.

[0024] Furthermore, the embodiment implements a heap hardware engine 114 that efficiently sorts the results from parallel first to nth similarity PE110a to 110n. The heap hardware engine 114 enables batch sorting using a device for a hardware-friendly implementation of the heap algorithm to handle multiple parallel queries and entries.

[0025] As illustrated, the query buffer 102 contains the first to the Nth queries 106a to 106n. The query buffer 102 includes a scheduler 104 (e.g., a finite state machine) that schedules queries consisting of the first to the Nth query vectors 106a to 106n to be streamed to the similarity PE 110. The query buffer 102 stores a set of query vectors 106 (e.g., queries). The entire candidate vector database can be sequentially compared against the query vectors 106.

[0026] The similarity search processor 116 (e.g., a dedicated processor and / or accelerator architecture) receives one query vector from the first to the Nth query vectors 106a to 106n at a time and performs similarity search processing. After processing this one query vector is complete, the similarity search processor 116 receives another query from the first to the Nth query vectors 106a to 106n and performs similarity search processing. As described below, the similarity search processor 116 stores a vector database and matches the vector database against the first to the Nth query vectors 106a to 106n. The vector database may be a large collection of feature vectors (e.g., a face database contains feature vectors extracted from each face of a large population). The feature vectors in the vector database are candidate vectors V000 to Vn03. The similarity search processor 116 may compare the first to Nth query vectors 106a to 106n (e.g., query faces) against candidate vectors V000 to Vn03 (e.g., faces) to identify the most similar matches (e.g., matches between faces).

[0027] The similarity search processor 116 includes a memory area 108 (e.g., a memory bank). In this example, the similarity search processor 116 may perform near-memory calculations using multiple similarity PEs 110 operating in parallel. For example, a vector database containing candidate vectors V000 to Vn03 may be stored in the memory area 108. The candidate vectors V000 to Vn03 may be of a size that can fit within the memory area 108, and thus all of the candidate vectors V000 to Vn03 can be efficiently contained in the memory area 108 (e.g., onboard memory). Therefore, each of the first to Nth memory areas 108a to 108n stores a vector of the number of "a"s from the vector database. Therefore, the similarity vectors PE110a to 110n for the 1st to Nth are accessed by a candidate vector V000 to Vn03 of the number "a*N", where N is the total number of memory areas 108a to 108n for the 1st to Nth.

[0028] As illustrated, each of the first to Nth memory areas 108a to 108n is connected to one of the first to Nth similarity PEs 110a to 110n. For example, the first memory area 108a is connected to and dedicated to the first similarity PE 110a, the second memory area 108b is connected to and dedicated to the second similarity PE 110b, and so on, and the Nth memory area 108n is connected to and dedicated to the Nth similarity PE 110n. For example, the first similarity PE 110a is prevented and / or prohibited from accessing the second to Nth memory areas 108b to 108n, which are dedicated to the other PEs among the second to Nth similarity PEs 110b to 110n. Therefore, each of the first to Nth similarity vectors PE110a to 110n has access to and operates only on a subset of the candidate vectors V000 to Vn03.

[0029] The scheduler 104 streams the query 106 to the similarity PE 110 in a daisy-chain manner. For example, the scheduler 104 may stream the first query vector 106a to the first similarity PE 110a (this may take several clock cycles). The first similarity PE 110a receives the first query vector 106a and then provides the first query vector 106a to the second similarity PE 110b over one or more clock cycles, and so on, until the Nth similarity PE 110n receives the first query vector 106a from a preceding similarity PE among the similarity PE 110s.

[0030] The first similarity PE110a may begin a similarity search when the first query vector 106a is received. For example, the first similarity PE110a may compare the first query vector 106a with the candidate vector V000 to determine the similarity between the first query vector 106a and the candidate vector V000. As described, the similarity may be determined by a distance metric such as the Manhattan distance, Euclidean distance, Minkowski distance, and Hamming distance, such that the similarity increases as the distance decreases. Each of the first to Nth similarity PE110a to 110n may perform a similar calculation for similarity when the first query vector 106a is received. For example, the second similarity PE110b may generate the distance between the first query vector 106a and the candidate vector V010, the Nth similarity PE110n may generate the distance between the first query vector 106a and the candidate vector Vn00, and so on for the rest.

[0031] Once the total distance is calculated, if it is less than the longest distance (described further below), the total distance may be transmitted to the hardware heap engine 114 and through the result engines 112, which are daisy-chained together. For example, after the first similarity PE 110a calculates the total distance between vector V000 and the first query vector 106a, the first similarity PE 110a transmits the total distance, associated with the vector ID of vector V000, to the first result engine 112a. Transmitting only the vector ID (rather than the entire vector V000) reduces bandwidth and facilitates identification at a later point in time. The first result engine 112a transmits the vector ID and total distance to the second result engine 112b, which then transmits the vector ID and total distance to subsequent result engines among the result engines 112, and so on, until the Nth result engine 112n is reached. The nth result engine 112n sends the vector ID and total distance to a buffer 114a (for example, an elastic buffer), which may temporarily store the vector ID and total distance until the heap controller 114b becomes available. The heap controller 114b may receive the vector ID and total distance when it becomes available. The heap controller 114b stores the vector ID and total distance in one of the nodes 0 to n (any number of nodes may be used) of the heap memory 114c.

[0032] Nodes 0 to n may be arranged as a tree data structure in some examples. For example, heap memory 114c may store nodes 0 to n as a binary tree data structure that holds the maximum (e.g., maximum heap binary tree) / minimum element (e.g., minimum heap binary tree) at the root of the tree. The configuration of the binary data tree structure may be set as maximum or minimum based on distance or similarity metric selection. Assuming that the maximum data structure is selected, the longest distance in this example is stored at the root. The longest distance retrieval unit 114d may retrieve and select the longest distance from nodes 0 to n. The longest distance retrieval unit 114d provides the longest distance to the 1st to Nth similarity PE110a to 110n. The longest distance may be the lowest similarity score (longest distance) from all similarity scores (distances) stored in heap memory 114c.

[0033] Each distance in the heap memory 114c reflects the similarity between one of the candidate vectors V000~Vn03 and the first query vector 106a. In this example, higher similarity corresponds to a shorter distance, while lower similarity corresponds to a longer distance. Thus, the longest distance is the distance between the candidate vectors V000~Vn03 that have the minimum similarity to the first query vector 106a, as identified by the heap memory 114c. The longest distance is used to perform parallel pruning to stop the analysis of the candidate vectors V000~Vn03 at an early stage.

[0034] For example, for a certain period of time, until the heap memory 114c is full, the first to Nth similarity PE110a to 110n may perform the entire distance calculation between the candidate vectors among the candidate vectors V000 to Vn03 and the first query vector 106a, and provide the total distance and vector ID to the hardware heap engine 114. If the heap memory 114c is full, the first to Nth similarity PE110a to 110n will perform a partial pruning process until the entire distance calculation is complete to determine whether or not to prune the calculation of the candidate vectors among the candidate vectors V000 to Vn03.

[0035] For example, suppose the longest distance retrieval unit 114d identifies that the longest distance stored in the heap memory 114c is "3". Further assume that the first similarity PE110a compares the first query vector 106a with the candidate vector V002. For example, the first query vector 106a may have 512 features, each of which is approximately 1 byte. Similarly, the candidate vector V002 may have 512 features, each of which is approximately 1 byte. The first similarity PE110a may compare features at the same index (e.g., byte) position to determine how similar those features are to each other and generate a distance based on the similarity. The first similarity PE110a accumulates the distances of the vector features compared so far in order to form a partial distance (e.g., the sum of the distances calculated so far, the running average of the distances calculated so far, the weighted sum of the distances calculated so far, etc.). The partial distance may be the distance accumulated in the ongoing distance calculation. For example, if the features of candidate vector V002 at byte positions 0-3 and the features of the first query vector 106a at byte positions 0-3 are compared and have associated distances, the partial distance will be the sum of those associated distances (e.g., partial distance = distance of the feature at byte value 0 + distance of the feature at byte value 1 + distance of the feature at byte value 2 + distance of the feature at byte value 3). It is worth noting that there can be 512 byte positions, and the partial distance reflects only the first part (the first 4 bytes) of those 512 byte positions. Thus, the partial distance is the running total of all distances calculated so far between the features of candidate vector V002 and the features of the first query vector 106a.

[0036] If the partial distance exceeds the longest distance received from the longest distance retrieval unit 114d, the first similarity PE110a may stop determining the similarity between candidate vector V002 and the first query vector 106a. For example, suppose the partial distance between candidate vector V002 and the first query vector 106a is a value of 4 (accumulated over the first 4 bytes), while the longest distance is a value of 3. It can already be concluded that candidate vector V002 has a higher degree of difference from the first query vector 106a than candidate vectors among candidate vectors V000~Vn03 that have already been analyzed and have associated distances stored in the heap memory 114c. In other words, the longest distance represents the highest degree of difference with the first query vector 106a in the heap memory 114c, and any analysis of any of the other candidate vectors among candidate vectors V000~Vn03 may be bypassed and ignored (pruned) if the partial distance of the other candidate vector exceeds the longest distance, regardless of how many of those other candidate vectors have been analyzed.

[0037] Doing so can save processing power and reduce latency. In this example, candidate vector V002 has been analyzed for four byte positions and has already accumulated partial distances exceeding the longest distance. Therefore, it is reasonable to conclude that candidate vector V002 is not a final similarity match for the first query vector 106a. Thus, the remaining bytes of candidate vector V002 do not need to be analyzed for similarity to the first query vector 106a, and the first similarity PE110a prioritizes analyzing other candidate vectors among candidate vectors V000~Vn03 and discards further analysis of candidate vector V002.

[0038] Furthermore, the first to Nth similarity PE110a to 110n operate cyclically to avoid stopping and waiting. If pruning occurs while calculating the distance over a feature at index n (e.g., byte position) for the previous candidate vector, the distance calculation for the next candidate vector begins at index n+1. This leverages the commutative and associative properties of distance calculation, ensuring that the calculated distance remains the same regardless of partial calculations starting from arbitrary feature indices. For example, candidate vector V002 is pruned based on a comparison of the features of candidate vector V002 at byte positions 0 to 3 with the features of the first query vector 106a. Therefore, the first similarity PE110a may have an index set at byte position 3. When the first similarity PE110a begins comparing the first query vector 106a with candidate vector V003, the first similarity PE110a does not need to reset its index to 0. Rather, the first similarity metric PE110a compares the features of candidate vector V003 with the features of the first query vector 106a at byte position 4 (index + 1), which is the next byte position after candidate vector V002 has been discarded. If the last byte position is reached, the first similarity metric PE110a may return to byte position 0 and determine the distance of candidate vector V003 at byte positions 0 to 3.

[0039] The first similarity PE110a may be iterated through all byte positions (including byte positions 0-4) of the candidate vector V003 and the first query vector 106a, as long as the partial distance does not exceed the longest distance. Assume that the partial distance does not exceed the longest distance, and therefore all 512 bytes of the candidate vector V003 are analyzed. The final distance may be the sum of all distances between the features of the first query vector 106a and the features of the candidate vector V003. That is, the features at all 512 byte positions of the first query vector 106a and the candidate vector V003 are compared and summed together to generate a total distance. If the total distance is less than the longest distance, the candidate vector V003 is determined to be in the list of the top K nearest neighbors for the first query vector 106a at that time and is provided to the hardware heap engine 114. Although the byte positions are described above, some embodiments may operate for different feature vector sizes (INT8, INT16, INT32, FP32, BF16) with different index positions.

[0040] The first similarity PE110a may transmit the vector ID and total distance of candidate vector V003 to the hardware heap engine 114 for storage. The heap controller 114b may store the vector ID and total distance of candidate vector V003 in the heap memory 114c and remove the vector ID associated with the longest distance and this longest distance. The longest distance retrieval unit 114d may select a new longest distance from the heap memory 114c and propagate this new longest distance to the first to Nth similarity PE110a to 110n. In particular, since the calculation of candidate vector V002 has been pruned, the first similarity PE110a does not transmit the partial distance and vector ID of candidate vector V002.

[0041] Therefore, each of the first to N similarity vectors PE110a to 110n may perform a partial distance analysis by comparing the partial distance of candidate vectors V000 to Vn03 with the longest distance, and stopping the analysis when the partial distance is greater than the longest distance. After all candidate vectors V000 to Vn03 have been analyzed for similarity to the first query vector 106a, the architecture 100 may output the results to the user or store the results. For example, the shortest distance and the corresponding node ID may be identified in the heap memory 114c. The final vector of candidate vectors V000 to Vn03 may be identified based on the corresponding node ID and output as the closest match to the first query vector 106a. In some examples, the application may request all K nearest neighbors / closest candidate matches for the query vector, where the maximum value of K is the size of the heap memory 114c (e.g., the number of nodes 0 to n). The hardware heap engine 114 may then return k distances and their corresponding node IDs to the application.

[0042] Subsequently, the heap controller 114b may remove all nodes from the heap memory 114c. The scheduler 104 may propagate the second query vector 106b to the first to Nth similarity levels PE110a to 110n. The first to Nth similarity levels PE110a to 110n may analyze the second query vector 106b for similarity with respect to candidate vectors V000 to Vn03, as described above. After the similarity analysis of the second query vector 106b is completed (comparison with all candidate vectors V000 to Vn03 is completed), other query vectors are sequentially streamed and analyzed until the processing of the last Nth query vector 106n is completed. In particular, throughout the entire streaming of query vectors 106, candidate vectors V000 to Vn03 remain within the memory area 108, avoiding high-latency memory access.

[0043] The memory area 108 may be an embedded static random access memory (SRAM) embedded within the similarity search processor 116 (e.g., on-chip). It is also worth mentioning that variations are possible in the heap memory 114c (e.g., minimum heap binary tree storage or maximum heap binary tree storage).

[0044] Referring here to Figures 2A and 2B, the architecture 300 for far-memory similarity search and matching is disclosed. Architecture 300 operates similarly to architecture 100 described above, and similar features are not described in detail for the sake of brevity. However, it will be understood that aspects of architecture 100 can be easily incorporated into architecture 300.

[0045] As shown in the diagram, a first processing array 324, a second processing array 326, and a third processing array 328 are provided. The first processing array 324, the second processing array 326, and the third processing array 328 may be located on the same SoC and / or form part of the same processor. Although the first processing array 324 is shown in detail, it will be understood that the second and third processing arrays 326, 328 consist of similar features and elements that are not shown for brevity. The first processing array 324, the second processing array 326, and the third processing array 328 may process different queries in parallel with each other.

[0046] In this example, the candidate vectors cannot be fully contained within the first to Nth memory areas 310 to 314 and are therefore retrieved and removed as needed. In this example, the transit buffer 304 and query buffer 306 receive both query and vector data from the fabric 302, which may be a network-on-chip fabric. The transit buffer 304 is sized appropriately for buffering candidate vectors from the candidate vector database (which may be stored in off-chip storage).

[0047] Each of the similarity PE316 is connected to a dedicated memory area of ​​memory area 310. Each of the memory areas 310 may operate as a "circular ping pong vector buffer" (CPPVB) that stores two database vectors in first and second buffers. To operate as a CPPVB, the first and second buffers of memory area 310 store multiple vectors at once. For example, suppose the first similarity PE316a has finished processing candidate vectors in one of the first and second buffers of the first memory area 310a. That buffer is replenished from row vector buffer 334, while the first similarity PE316a processes candidate vectors in the other of the first and second buffers.

[0048] The similarity PE316 is connected in a 1D systolic array manner. The results from the similarity PE316 are daisy-chained through the results engine 318 to form a single stream of results that is sent to the multiplexer (MUX) 320a. The MUX320a may provide those results to the MUX320b, which in turn provides those results to the shared hardware heap engine (SHHE) 336. The SHHE336 is discussed in more detail with respect to Figure 2B, which illustrates the SHHE336 in detail.

[0049] Multiple similarity PE316s may compute similarity searches for the same query (e.g., the same query vector). The query streaming mechanism of the first query buffer 306a is the same as that of the query buffer 102 (Figure 1) and will not be repeated in detail. As mentioned above, each similarity PE316 is connected to one of the first to Nth memory areas 310a to 310n, where the memory area contains a CPPVB. For example, the first memory area 310a contains a CPPVB containing the first and second buffers. The first and second buffers store two vectors from the database for comparison with the query. Each feature in the vectors stored in the first and second buffers is iteratively streamed to the first similarity PE316a.

[0050] For example, initially, all similarity PE316s begin calculations on the first feature in the associated first buffer. In each clock cycle, the similarity PE316 receives a buffer stream from the first buffer, determines the similarity between the i-th feature of the candidate vector and the query vector, and determines the partial distance. If the candidate vector calculation is pruned, the query stream index is not interrupted. For example, SHHE336 may provide the longest distance. Suppose the first similarity PE316a determines that the first candidate vector in the first buffer of the first memory area 310a will be discarded at feature index "i" (e.g., byte position) (e.g., based on the partial distance being greater than the longest distance). The first similarity PE316a accesses the second buffer of the first memory area 310a to retrieve the second candidate vector from the second buffer without interrupting the flow of streaming query bytes. The first similarity analyzer, PE316a, starts with feature index "i+1" and analyzes the similarity between the second candidate vector and the query vector, continuing in a cyclical manner until all features have been processed or the calculation of the second vector has been pruned.

[0051] While the first similarity PE316a performs similarity analysis based on the second candidate vector, the row vector buffer 334 may store the third candidate vector in the first buffer of the first memory area 310a. After processing the second vector is complete or the calculation of the second vector has been pruned, the first similarity PE316a switches to the first buffer and starts similarity analysis on the third candidate vector. The row vector buffer 334 may start storing the fourth candidate vector for the first similarity PE316a in the second buffer for analysis. Thus, the first similarity PE316a switches between the first and second buffers in a ping-pong fashion. The second to Nth similarity PE316b to 316n may similarly access candidate vectors from the first and second buffers of the second to Nth memory areas 310b to 310n in a ping-pong fashion.

[0052] Transit buffer 304 fetches a vector database (e.g., 1 billion candidate vectors) from system memory (not shown). The candidate vectors in transit buffer 304 are broadcast to all row vector buffers of the first, second, and third processing arrays 324, 326, and 328, which include row vector buffer 334. The first and second buffers fetch candidate vectors from row vector buffer 334. Since the first to N similarity measures PE316a to 316n calculate similarity measurements for the same query, the candidate vectors stored in the first and second buffers are different and mutually exclusive. It is worth noting that each of the first, second, and third processing arrays 324, 326, and 328 may store the same candidate vectors.

[0053] For example, each of the first, second, and third processing arrays 324, 326, and 328 may receive the first candidate vector and perform similarity analysis on the first candidate vector. If all of the first, second, and third processing arrays 324, 326, and 328 have received the first candidate vector, the transit buffer 304 removes the first candidate vector from its memory and replaces it with a new vector from the fabric 302 (for example, the second candidate vector). The row vector buffers, including the row vector buffer 334, may then provide the new vector to the first, second, and third processing arrays. Vector fetching by the transit buffer 304 and pulling by row vector buffers such as the row vector buffer 334 are performed continuously until the entire database has been analyzed for similarity.

[0054] As described above, the second and third processing arrays 326 and 328 are composed of components similar to those of the first processing array 324. The second query buffer 306b may provide queries to the second processing array 326. The third query buffer 306c may provide queries to the third processing array 328.

[0055] In architecture 300, the first, second, and third processing arrays 324, 326, and 328 perform similarity searches in parallel across multiple different queries. SHHE336 operates similarly to the hardware heap engine 114 (Figure 1), but with the additional capability to handle data related to several separate query searches and perform pruning based on partial distance, as described above. The results from each of the first to N similarity PE316a to 316n are daisy-chained and transmitted through the first to N result engines 318a to 318n. Each of the first to N result engines 318a to 318n may include two buffers to store results in case of backpressure or delays in the transmission of results through the first to N result engines 318a to 318n.

[0056] The outputs from the first, second, and third processing arrays 324, 326, and 328 are provided to MUX320a and 320b to generate a single stream of results from the first, second, and third processing arrays 324, 326, and 328. Figure 2B shows a more detailed diagram of the SHHE336, with the relevant components from Figure 2A also shown. Referring to Figure 2B, to avoid collisions and backpressure when writing to the SHHE336, the result stream may be stored in a large-size buffer 332 (e.g., a 64-depth elastic buffer). The heap controller 330 reads the candidate vector IDs, corresponding query IDs, and corresponding distances stored in the buffer 332 and controls the insertion flow into the corresponding partitions for the query IDs.

[0057] For example, the first, second, and third processing arrays 324, 326, and 328 may provide output including a candidate vector ID corresponding to a candidate vector being compared to a query vector, the total distance associated with the candidate vector and the query vector, and a query ID corresponding to the query vector. The query ID may be referenced to determine whether the candidate vector and its corresponding total distance should be stored in the first heap memory 322a, the second heap memory 322b, or the third heap memory 322c of the heap memory 322. The first heap memory 322a may store the results from the first processing array 324 associated with the first query in nodes 0 to n. The second heap memory 322b may store the results from the second processing array 326 associated with the second query in nodes 0 to n. The third heap memory 322c may store the results from the third processing array 328 associated with the third query in nodes 0 to n.

[0058] For example, suppose the second processing array 326 analyzes the first candidate vector for similarity to the second query vector. The second processing array 326 may determine that the total distance of the first candidate vector is less than the longest distance associated with the second query vector. Therefore, the second processing array 326 may provide MUX320a with an output including the first candidate vector ID, the second query ID (associated with the second query vector), and the total distance. MUX320a and 320b may provide this output to SHHE336. SHHE336 receives this output and may store it in buffer 332 until the heap controller 330 is ready to store it. The heap controller 330 receives this output and may extract the second query ID. The second query ID may correspond to the second heap memory 322b. That is, each result associated with the second query vector may be stored in the second heap memory 322b. Therefore, the heap controller 330 may associate the first candidate vector ID and distance with each other and store them in the second heap memory 322b. Thus, the heap controller 330 may identify the query ID in order to determine where to store the candidate vector ID and distance.

[0059] The longest distance retrieval unit 338 further determines the longest distance from the first heap memory 322a. The longest distance to the first heap memory 322a may be the longest distance of the first query stored in the first heap memory 322a. The longest distance retrieval unit 338 further provides the longest distance of the first query to the first processing array 324. The longest distance retrieval unit 338 further determines the longest distance from the second heap memory 322b. The longest distance to the second heap memory 322b may be the longest distance of the second query stored in the second heap memory 322b. The longest distance retrieval unit 338 further provides the longest distance of the second query to the second processing array 326. The longest distance retrieval unit 338 further determines the longest distance from the third heap memory 322c. The longest distance in the third heap memory 322c may be the longest distance of the third query stored in the third heap memory 322c. The longest distance retrieval unit 338 further provides the longest distance of the third query to the third processing array 328. The first processing array 324 may perform a pruning process based on the longest distance of the first query. Similarly, the second and third processing arrays 326 and 328 may perform a pruning process based on the longest distances of the second and third queries, respectively.

[0060] Figure 3 shows Method 800 of a similarity search process using pruning. Method 800 may generally be implemented using embodiments described herein, for example, the previously discussed architecture 100 (Figure 1) and / or architecture 300 (Figures 2A-2B). In one embodiment, Method 800 is implemented in one or more modules as a set of logic instructions stored in a machine or computer-readable storage medium such as random access memory (RAM), read-only memory (ROM), programmable ROM (PROM), firmware, flash memory, etc., in configurable logic such as a programmable logic array (PLA), field-programmable gate array (FPGA), complex-programmable logic device (CPLD), for example, in fixed-function logic hardware using circuit technologies such as application-specific integrated circuits (ASICs), complementary metal-oxide-semiconductor (CMOS) or transistor-transistor logic (TTL) technology, or in any combination thereof.

[0061] For example, computer program code that performs the operations shown in Method 800 may be written in any combination of one or more programming languages, which include object-oriented programming languages ​​such as Java®, Smalltalk®, and C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. In addition, logic instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, state setting data, configuration data for integrated circuits, state information for personalizing electronic circuits, and / or other structural components specific to hardware (e.g., host processor, central processing unit / CPU, microcontroller, etc.).

[0062] The illustrated processing block 802 uses a first processing engine among multiple processing engines to determine a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector. The illustrated processing block 804 uses a second processing engine among multiple processing engines to determine a total similarity measure based on the query vector and a second candidate vector. The illustrated processing block 806 uses the first processing engine to determine whether or not to compare a second portion of the query vector with a second portion of the first candidate vector, based on the first partial similarity measure and the total similarity measure.

[0063] In some embodiments, Method 800 further comprises the step of using a first processing engine to compare a second portion of a query vector with a second portion of a first candidate vector in response that a first partial similarity measure is less than a total similarity measure, where the first partial similarity measure is a partial distance and the total similarity measure is a total distance. In some examples, Method 800 further comprises the steps of using multiple processing engines to find different candidate vectors, using multiple processing engines to determine multiple partial similarity measures between a first portion of a query vector and a first portion of a different candidate vector, and using multiple processing engines to determine whether to bypass the partial similarity calculation between a second portion of a query vector and a second portion of a different candidate vector based on the multiple partial similarity measures and total similarity measures.

[0064] In some examples, Method 800 further comprises a step of accessing multiple memory storage areas, each dedicated to one of multiple processing engines. The multiple memory storage areas store different candidate vectors. The different candidate vectors represent a vector candidate database. In some examples, Method 800 further comprises a step of using a first processing engine to determine whether to bypass the similarity calculation of the first candidate vector based on first partial similarity measures and gross similarity measures. The index for the query vector is the value at which the first partial similarity measure is determined. In response to the bypass of the similarity calculation of the first candidate vector, Method 800 uses the first processing engine to increment the value of the index and uses the first processing engine to determine whether to bypass the similarity calculation of the third candidate vector based on partial similarity measures that will be determined based on the feature values ​​of the third candidate vector and the query vector, both of which are associated with the incremented value of the index.

[0065] In some examples, method 800 further comprises the step of storing a total similarity measure and a plurality of similarity measures in a maximum heap binary tree or a minimum heap binary tree. The plurality of similarity measures are determined based on different candidate vectors and query vectors. The total similarity measure is greater than each of the plurality of similarity measures. In some examples, method 800 further comprises the step of storing a plurality of candidate vectors in a plurality of ping-pong buffers; the step of determining a plurality of partial similarity measures based on a first portion of a plurality of query vectors and a first portion of a plurality of candidate vectors using a plurality of processing engines; and the step of determining, using a plurality of processing engines, that similarity calculations associated with a first subset of the plurality of candidate vectors are to be bypassed based on a first subset of the plurality of partial similarity measures and a first total similarity measure.

[0066] In some examples, Method 800 further comprises the step of determining, using a group of processing engines, that a second subset of candidate vectors will be processed based on a second subset of partial similarity measures and a first total similarity measure. Method 800 further comprises the step of determining a second total similarity measure based on a second subset of candidate vectors and query vectors using a group of processing engines, and storing each of the second total similarity measures in a different heap memory based on the identification information of one of the query vectors associated with each of the query vectors. Each of the different heap memories is dedicated to one of the query vectors.

[0067] Figure 4 shows the timing diagram 400 for PE0 402 and PE0n 404. PE0 402 and PE0n 404 can be easily substituted for either similarity PE110 (Figure 1) or similarity PE316 (Figures 2A-2B). PE0 402 and PE0n 404 operate on a single query with 512 features, each having a size of 1 byte. The 512 bytes in the query are represented as Q0, Q1...Q511 in the timing diagram 400. The diagram presents two cases: one is pruning the calculation for PE0 402, and the other is accepting the calculation for PE0n 404.

[0068] At the start of timing diagram 400, referring to signal 406, PE0 402 compares the query vector Q with candidate vector V00 from memory bank 0 (indicated in the buffer / vector number row). In the 226th clock cycle (performed at Q226), the accumulated partial distance 226 (shaded in the background) for v00 is greater than the maximum distance (which may be received from the hardware heap engine or SHHE), and the similarity calculation for V00 is pruned (e.g., terminated) at index 226. In the next clock cycle, candidate vector v01 is loaded into PE0 402, and Q227 from the query vector is used to calculate the similarity distance from candidate vector v01 to the 227th feature (V227 at the index + 1 position above). Therefore, PE0 402 begins the distance calculation for the next database candidate vector v01. PE0 402 continues the similarity calculation cyclically up to index 511, and then down to indices 0, 1...226, unless pruning is performed and the calculation is dropped.

[0069] The query stream is shared between PE0 402 and PE0n 404. Therefore, both PE0 402 and PE0n 404 operate on the same query vector. In the lower signal 408 of timing diagram 400, PE0n 404 compares the query byte Q0 with candidate vector vn0 with an offset of n clock cycles relative to PE0 402 over the time difference between time point B and time point A. This is partly due to the daisy-chain transmission of the query vector through the entire PE, including PE0 402 and PE0n 404. Therefore, PE0n 404 receives the query vector after PE0 402. PE0n 404 calculates the partial distance of the query feature and candidate vector vn0 to the corresponding vector feature over 512 clock cycles, not exceeding the longest distance. After the 512th clock cycle (or the 511th clock cycle in the case of a 0-indexed index), the partial distance is still less than the longest distance. Therefore, candidate vector vn0 is recognized as a candidate for the K nearest neighbor for the query vector and is consequently pushed to the hardware heap engine or SHHE for storage. In the next clock cycle, PE0n 404 selects feature Q0 from the query vector. PE0n 404 selects feature 0 (V0) from the new candidate vector vn1 and begins calculating the partial distance.

[0070] Figure 5 shows the query streaming method 500. Method 500 may generally be implemented using embodiments described herein, for example, the architecture 100 (Figure 1), architecture 300 (Figures 2A-2B), and / or timing diagram 400 (Figure 4), which have already been discussed. For example, Method 500 may be executed by a scheduler 104 of the query buffer 102 (Figure 1) and / or query buffer 306 (Figures 2A-2B). Method 500 may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc., in configurable logic such as PLA, FPGA, CPLD, etc., in fixed-function logic hardware using circuit technology such as ASIC, CMOS, or TTL technology, etc., or in any combination thereof.

[0071] The illustrated processing block 502 streams the selected query vector to multiple similarity PEs. The illustrated processing block 504 determines whether the selected query vector has been compared to all candidate vectors. If it has not been compared, processing block 502 is executed. If it has been compared, the illustrated processing block 506 determines whether it has been completed for all query vectors. If it has not been completed, the illustrated processing block 508 selects a new query vector as the selected query vector, processing block 502 is executed to process the selected query vector, and similarity measurements of the selected query vector to the candidate vectors are calculated.

[0072] Figure 6 shows a similarity calculation method 530 implemented by similarity PE. Method 530 may generally be implemented using embodiments described herein, for example, the previously discussed architecture 100 (Figure 1), architecture 300 (Figures 2A-2B), timing diagram 400 (Figure 4), and / or method 500 (Figure 5). For example, method 530 may be performed by similarity PE110 (Figure 1) and / or similarity PE316 (Figure 2A). Method 530 may be implemented in one or more modules as a set of logic instructions stored in a mechanical or computer-readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc., in configurable logic such as PLA, FPGA, CPLD, etc., in fixed-function logic hardware using circuit technology such as ASIC, CMOS, or TTL technology, etc., or in any combination thereof.

[0073] The illustrated processing block 542 sets the index value to 0. The index value may be the byte position (or corresponding to the feature vector sizes of the query and candidate vectors) that the similarity PE refers to in order to compare the feature values ​​of the candidate vector and the query vector at a certain byte position and determine the similarity measurement. That is, the similarity PE first starts by fetching address 0 for the candidate and query vectors. Therefore, initially, both the query vector and the candidate vector start at index 0.

[0074] The illustrated processing block 532 calculates the feature distance between the features of the query vector at the index value and the features of the candidate vector at the index value. The illustrated processing block 534 adds the feature distance to the partial distance to generate a sum, and sets this sum as the new partial distance. In some examples, processing block 534 calculates the average of the distances calculated so far or a weighted sum of the distances calculated so far, and sets that value as the partial distance. The illustrated processing block 536 determines whether the partial distance is greater than the longest distance.

[0075] If a partial distance is greater than the longest distance, the remainder of the calculation is pruned and discarded for that candidate vector. For example, the illustrated processing block 544 determines whether any further candidate vectors exist. If so, the illustrated processing block 546 selects a new candidate vector from the remaining candidate vectors and sets its partial distance to 0. Processing block 548 increments the index value. Processing block 532 is then executed.

[0076] If processing block 536 determines that the longest distance is greater than the partial distance, processing block 538 (shown in the diagram) determines whether the last feature in the candidate vector has been reached. If not, processing block 540 (shown in the diagram) increments the index value, and as a result, processing block 532 calculates the vector distance of the feature at the incremented index value, and so on for the rest. If processing block 538 determines that the last feature in the candidate vector has been reached, processing block 542 (shown in the diagram) pushes the result to the sorting engine (e.g., hardware heap engine or SHHE). That is, if the partial distances for all features have been accumulated and the final total distance (which is the sum of all partial distances) is still less than the longest distance, the result is sent to the sorting engine.

[0077] Figure 7 shows a heap memory structure 550 which is a binary tree. The heap memory structure 550 may generally be implemented using embodiments described herein, for example, the previously discussed architecture 100 (Figure 1), architecture 300 (Figures 2A-2B), timing diagram 400 (Figure 4), method 500 (Figure 5), and / or method 600 (Figure 6). For example, nodes 0-n of heap memory 114c (Figure 1), nodes 0-n of the first heap memory 322a (Figure 2B), nodes 0-n of the second heap memory 322b (Figure 2B), and / or nodes 0-n of the third heap memory 322c (Figure 2B) may be organized into the heap memory structure 550.

[0078] The heap memory structure 550 may include nodes 1 to 15 organized into a heap structure, numbered from the root (node ​​1) to the leaves (nodes 8 to 15) (e.g., a pre-order sequence). The node numbering corresponds to a storage location (node ​​index) within the hardware heap engine. The hardware heap engine partitions common memory to store K nearest neighbors (KNN) for batches of queries, where K and batch size are configurable. The number of nodes (15) is illustrative, and embodiments as described herein may include any number of nodes and may be determined for a plurality of KNN values ​​to be stored (e.g., 20 KNN values ​​would result in 20 nodes).

[0079] In structure 550, the heap memory structure 550 is configured to store 15 nearest vectors in each partition for a single query. The heap binary structure may be a max heap binary tree, in which the root node 1 has the longest distance value, the first level (i.e., nodes 2 and 3) has the next longest distance value, the second level (i.e., nodes 4-7) has the next longest distance value, and the lowest level (i.e., nodes 8-15) has the shortest distance value. As will be described in more detail, a max heap binary tree can be an efficient memory structure because the longest distance is always maintained and the root of the tree can be easily identified.

[0080] Furthermore, the insertion of new values ​​into the tree can be performed efficiently. For example, if a new distance value is to be inserted into structure 550, the distance value (longest distance) in node 1 is automatically removed. The new distance value may be compared with the distance value of node 2. If the distance heap value of node 2 is greater than the new distance value, the distance value (and corresponding candidate vector ID) in node 2 may be moved to node 1, and the new distance value may occupy node 2. Subsequently, the new distance value is compared with the distance of one child node of node 2 (nodes 4 and 5), and if the new distance is less than the distance of that child node, it is swapped with that child node. This process may be repeated until the new distance is no longer smaller than the child node of the node currently occupying the new distance, or until the position of the new distance reaches the bottom level of the maximum heap binary tree. In particular, the new distance does not need to be compared with all distances of nodes 2-15, and only (at most) three comparisons need to be performed to find the final position. In other words, there is no need for a strict ordering of the distances from maximum to minimum; only the maximum distance must be identified, which is stored in node 1. Furthermore, each of nodes 1 through 15 may contain a candidate vector ID corresponding to the distance value stored in that node (for example, the candidate vector ID of a candidate vector that underwent a similarity calculation process to generate the distance value stored in the node).

[0081] Structure 550 may be duplicated for each partition, query memory, or query. A copy of the root of node 1 is stored in a register in hardware and broadcast as the longest distance to the appropriate similarity PE that operates for each query in order to compare and remove redundant results.

[0082] In some examples, if distance calculations are not pruned / dropped, the distance results are daisy-chained to a hardware heap engine (HHE), which creates structure 550. The HHE is a device for a hardware-friendly implementation of a conventional heap. A heap, particularly a max heap (or max-heap) binary tree, efficiently stores distances for k "nodes" and allows easy access to the maximum distance from the root node 1. A characteristic of the max heap is that the values ​​within a node must be greater than those of its child nodes, while in the case of the minimal heap, the values ​​within a node must be less than those of its child nodes. Therefore, the root node 1 of structure 550 stores the largest element in the data structure 550. In some embodiments, the HHE may be configured to run as either a max heap or a minimal heap.

[0083] Figure 8 shows Method 420 implemented by HHE and / or SHHE to fill an incomplete (not yet fully filled) structure (e.g., a binary tree). Method 420 may generally be implemented using embodiments described herein, e.g., Architecture 100 (Figure 1), Architecture 300 (Figures 2A-2B), Timing Diagram 400 (Figure 4), Method 500 (Figure 5), Method 530 (Figure 6), and / or Structure 550 (Figure 7), which have already been discussed. For example, Method 420 may generate Structure 550 (Figure 7). Method 420 may be implemented in one or more modules as a set of logic instructions stored in a mechanical or computer-readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc., in configurable logic such as PLA, FPGA, CPLD, etc., in fixed-function logic hardware using circuit technology such as ASIC, CMOS, or TTL technology, etc., or in any combination thereof.

[0084] The illustrated processing block 422 identifies a new node entry. The illustrated processing block 424 stores the new node entry in the first available location, starting from index 1 (for example, downward from the root node to lower levels of the hierarchy). The illustrated processing block 426 determines whether the new node's location is the root node. If it is the root node, no further action is required. If the current location is not the root node, the current location is a child node. Therefore, the illustrated processing block 428 determines whether the distance value of the new node entry (stored in the child node) is greater than the distance of the parent node of the new node's location (child node). If it is not greater, no further action is required. If it is greater, and the distance of the current node is greater than the distance of the parent node, the illustrated processing block 430 moves the new node entry to the parent node and moves the parent node entry to the child node. That is, the illustrated processing block 430 swaps the parent node entry with the new node entry in the child node. Subsequently, the illustrated processing block 426 is executed again, with the new node's location set to the parent node's location.

[0085] Figure 9 shows Method 440 implemented by HHE and / or SHHE for inserting a new node entry into a full structure (e.g., a binary tree with all nodes occupied). Method 440 may generally be implemented using embodiments described herein, such as the already discussed architecture 100 (Figure 1), architecture 300 (Figures 2A-2B), timing diagram 400 (Figure 4), method 500 (Figure 5), method 530 (Figure 6), structure 550 (Figure 7), and / or method 420 (Figure 8). For example, Method 440 may update structure 550 (Figure 7). Method 440 may be implemented in one or more modules as a set of logic instructions stored in a mechanical or computer-readable storage medium such as RAM, ROM, PROM, firmware, or flash memory; in configurable logic such as PLA, FPGA, or CPLD; in fixed-function logic hardware using circuit technology such as ASIC, CMOS, or TTL technology; or in any combination thereof.

[0086] The illustrated processing block 442 inserts a new node entry (with a total distance less than the longest distance of the head or root node) into the head node of the maximum heap binary tree. The entry that was immediately present in the head of the maximum heap binary tree is deleted and removed from the maximum heap binary tree. The illustrated processing block 444 reads the distances of the parent node's left and right child nodes. The illustrated processing block 446 determines whether the distance of the right child node is greater than the distance of the left child node. If it is, the illustrated processing block 454 determines whether the distance of the right child node is greater than the distance of the parent node. If it is not greater, method 440 terminates. If processing block 454 determines that the distance of the right child node is greater than the distance of the parent node, the illustrated processing block 450 swaps the node entry of the parent node with the entry in the right child node. The illustrated processing block 452 determines whether the right child node is a leaf node (the lowest level of the binary tree). If it is a leaf node, method 440 terminates. If it is not a leaf node, the illustrated processing block 460 sets the right child node as the parent node, and processing block 444 is executed.

[0087] If processing block 446 determines that the distance of the right child node is not greater than the distance of the left child node, the illustrated processing block 448 is executed. Processing block 448 determines whether the distance of the left child node is greater than the distance of the parent node. If it is not greater, method 440 terminates. If processing block 448 determines that the distance of the left child node is greater than the distance of the parent node, the illustrated processing block 456 swaps the node entry of the parent node with the entry in the left child node. The illustrated processing block 458 determines whether the left child node is a leaf node. If it is not a leaf node, the illustrated processing block 462 sets the left child node as the parent node. If it is a leaf node, method 440 terminates.

[0088] Referring here to Figure 10, a similarity search and pruning query processing computing system 158 is shown. System 158 may generally be part of an electronic device / platform having computing functions (e.g., personal digital assistants / PDAs, notebook computers, tablet computers, convertible tablets, servers), communication functions (e.g., smartphones), imaging functions (e.g., cameras, camcorders), media playback functions (e.g., smart TVs / TVs), wearable functions (e.g., watches, glasses, hats, footwear, jewelry), vehicle functions (e.g., cars, trucks, motorcycles), robotic functions (e.g., autonomous robots), etc., or any combination thereof. In the illustrated example, system 158 includes a host processor 160 (e.g., CPU) having an integrated memory controller (IMC) 154 coupled to a system memory 164.

[0089] The illustrated system 158 also includes an input / output (IO) module 166 implemented on a semiconductor die 170 as a system-on-a-chip (SoC), along with a host processor 160, a graphics processor 162 (e.g., a GPU), a similarity search processor 150, a ROM 140, and an AI accelerator 148. The illustrated IO module 166 communicates with, for example, a display 172 (e.g., a touchscreen, a liquid crystal display / LCD, a light-emitting diode / LED display), a network controller 174 (e.g., wired and / or wireless), an FPGA 178, and mass storage 168 (e.g., a hard disk drive / HDD, an optical disk, a solid-state drive / SSD, flash memory). Furthermore, the SoC 170 may further include a processor (not shown) and / or an AI accelerator 148 dedicated to artificial intelligence (AI) and / or neural network (NN) processing. For example, the system SoC 170 may include a visual processing unit (VPU) and / or other AI / NN-specific processors, such as the AI ​​accelerator 148. In some embodiments, any aspect of the embodiments described herein may be implemented in a processor and / or accelerator dedicated to AI and / or NN processing, such as the AI ​​accelerator 148, graphics processor 162, host processor 160 and / or similarity search processor 150.

[0090] The similarity search processor 150 may execute instructions 156 retrieved from system memory 164 (e.g., dynamic random access memory) and / or mass storage 168 to implement embodiments as described herein. The similarity search processor 150 performs batch processing, similarity search of candidate vectors against query vectors, and early pruning of candidate vector calculations in PE1-PE. n It may include 152. To do this, some examples are PE1~PE nCandidate vectors may be stored in a memory storage area 144 having multiple partitions, one of which is dedicated to one of the 152. If the candidate vectors are too large to fit within the memory storage area 144, a subset of the candidate vectors may be stored in PE1~PE n 152 may be stored in a ping-pong buffer 142 (e.g., static random access memory) that is accessed to compare the query vector with a subset of the candidate vectors. The query and candidate vectors may be stored in mass storage 168 when not in use and moved to memory storage area 144, ping-pong buffer 142 and / or system memory 164 when a similarity search is to be performed. When instruction 156 is executed, the computing system 158 may implement one or more aspects of the embodiments described herein. For example, system 158 may implement one or more aspects of the previously discussed architecture 100 (Figure 1), architecture 300 (Figures 2A-2B), timing diagram 400 (Figure 4), method 500 (Figure 5), method 530 (Figure 6), structure 550 (Figure 7), method 420 (Figure 8) and / or method 440 (Figure 9). Therefore, the illustrated computing system 158 is considered to have improved performance to the extent that it enables the computing system 158 to utilize low-latency similarity search and pruning processes to reduce processing power, overhead, and far memory access. In some examples, the memory storage area 144 may include, if desired, a ping-pong buffer 142.

[0091] Figure 11 shows a semiconductor device 180 (e.g., a chip, die, or package). The illustrated device 180 includes one or more substrates 184 (e.g., silicon, sapphire, or gallium arsenide) and logic 182 (e.g., a transistor array and other integrated circuit / IC components) coupled to the substrates 184. In one embodiment, the device 180 is operated during the application development phase, and the logic 182 implements one or more aspects of the already discussed architecture 100 (Figure 1), architecture 300 (Figures 2A-2B), timing diagram 400 (Figure 4), method 500 (Figure 5), method 530 (Figure 6), structure 550 (Figure 7), method 420 (Figure 8), and / or method 440 (Figure 9). Therefore, logic 182 may use a first processing element among a plurality of processing elements to determine a first partial similarity measure based on a first portion of the query vector and a first portion of the first candidate vector, use a second processing element among a plurality of processing elements to determine a total similarity measure based on the query vector and the second candidate vector, and use the first processing element to determine whether or not to compare a second portion of the query vector with a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure. Furthermore, logic 182 may further include a processor (not shown) and / or an AI accelerator dedicated to artificial intelligence (AI) and / or NN processing. For example, system logic 182 may include a VPU and / or other AI / NN-specific processors, such as an AI accelerator, a similarity search PE, etc. In some embodiments, any aspect of the embodiments described herein may be implemented in a processor and / or accelerator dedicated to AI and / or NN processing, such as an AI accelerator.

[0092] Logic 182 may be implemented in at least partially configurable logic or fixed-function hardware logic. In one example, logic 182 includes a transistor channel region located (e.g., embedded) within the substrate 184. Therefore, the interface between logic 182 and the substrate 184 does not have to be a step junction. Logic 182 may also be considered to include an epitaxial layer growing on the initial wafer of the substrate 184.

[0093] Figure 12 shows a processor core 200 according to one embodiment. The processor core 200 may be the core of any type of processor, such as a microprocessor, an embedded processor, a digital signal processor (DSP), a network processor, or any other device that executes code. Although only one processor core 200 is shown in Figure 12, the processing elements may alternatively include more than one of the processor cores 200 shown in Figure 12. The processor core 200 may be a single-threaded core, or, in at least one embodiment, the processor core 200 may be multi-threaded in the sense that it may include more than one hardware thread context (or “logical processor”) per core.

[0094] Figure 12 also shows a memory 270 coupled to the processor core 200. The memory 270 may be any of the wide variety of memories (including various layers of the memory hierarchy) that are known to those skilled in the art or otherwise available to those skilled in the art. The memory 270 contains one or more code 213 instructions executed by the processor core 200, and the code 213 may implement one or more aspects of embodiments such as the already discussed architecture 100 (Figure 1), architecture 300 (Figures 2A-2B), timing diagram 400 (Figure 4), method 500 (Figure 5), method 530 (Figure 6), structure 550 (Figure 7), method 420 (Figure 8), and / or method 440 (Figure 9). The processor core 200 follows a program sequence of instructions indicated by the code 213. Each instruction is input to the front-end unit 210 and processed by one or more decoders 220. The decoder 220 may generate microoperations such as fixed-width microoperations in a predefined format as its output, or it may generate other instructions, microinstructions, or control signals that reflect the original code instructions. The illustrated front-end unit 210 also includes register renaming logic 225 and scheduling logic 230, which generally allocate resources and queue operations corresponding to translation instructions for execution.

[0095] The processor core 200 is shown as including execution logic 250 having a set of execution units 255-1 to 255-N. Some embodiments may include multiple execution units dedicated to a particular function or set of functions. Other embodiments may include only one execution unit, or one execution unit capable of performing a particular function. The illustrated execution logic 250 performs operations specified by code instructions.

[0096] After the execution of the operation specified by the code instruction is complete, the backend logic 260 retires the instruction of code 213. In one embodiment, the processor core 200 enables out-of-order execution but requests in-order retirement of the instruction. The retirement logic 265 can take various forms known to those skilled in the art (e.g., a reorder buffer). In this way, the processor core 200 is transformed during the execution of code 213, at least in terms of the output generated by the decoder, the hardware registers and tables utilized by the register renaming logic 225, and any registers (not shown) modified by the execution logic 250.

[0097] Although not shown in Figure 12, the processing element may include other elements on the chip having the processor core 200. For example, the processing element may include memory control logic together with the processor core 200. The processing element may include I / O control logic and / or I / O control logic integrated with the memory control logic. The processing element may also include one or more caches.

[0098] Referring now to Figure 13, a block diagram of an embodiment of a computing system 1000 according to one embodiment is shown. Figure 13 shows a multiprocessor system 1000 comprising a first processing element 1070 and a second processing element 1080. Although two processing elements 1070 and 1080 are shown, it will be understood that one embodiment of system 1000 may also include only one such processing element.

[0099] System 1000 is shown as a point-to-point interconnect system, where the first processing element 1070 and the second processing element 1080 are connected via the point-to-point interconnect 1050. It should be understood that any or all interconnects shown in Figure 13 may be implemented as multidrop buses rather than point-to-point interconnects.

[0100] As shown in Figure 13, each of the processing elements 1070 and 1080 may be a multicore processor including a first processor core and a second processor core (i.e., processor cores 1074a and 1074b, and processor cores 1084a and 1084b). Such cores 1074a, 1074b, 1084a, and 1084b may be configured to execute instruction codes in a manner similar to that discussed above in relation to Figure 12.

[0101] Each processing element 1070, 1080 may include at least one shared cache 1896a, 1896b. The shared caches 1896a, 1896b may store data (e.g., instructions) used by one or more components of the processor, such as cores 1074a, 1074b and 1084a, 1084b, respectively. For example, the shared caches 1896a, 1896b may locally cache data stored in memory 1032, 1034 for faster access by the processor components. In one or more embodiments, the shared caches 1896a, 1896b may include one or more intermediate-level caches, last-level caches (LLCs), and / or combinations thereof, such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other level caches.

[0102] Although only two processing elements 1070 and 1080 are shown, it is understood that the scope of the embodiments is not limited thereto. In other embodiments, one or more additional processing elements may be present within a given processor. Alternatively, one or more of the processing elements 1070 and 1080 may be non-processor elements such as accelerators or field-programmable gate arrays. For example, the additional processing elements may include an additional processor identical to the first processor 1070, an additional processor heterogeneous or asymmetrical to the first processor 1070, an accelerator (e.g., a graphics accelerator or a digital signal processing (DSP) unit), a field-programmable gate array, or any other arbitrary processing element. A variety of differences may exist between the processing elements 1070 and 1080 in terms of various merit metrics, including architectural characteristics, microarchitectural characteristics, thermal characteristics, power consumption characteristics, etc. These differences may effectively manifest as asymmetry and heterogeneity between the processing elements 1070 and 1080. In at least one embodiment, various processing elements 1070, 1080 may be located within the same die package.

[0103] The first processing element 1070 may further include memory controller logic (MC) 1072, and point-to-point (PP) interfaces 1076 and 1078. Similarly, the second processing element 1080 may include MC 1082, and PP interfaces 1086 and 1088. As shown in Figure 13, MCs 1072 and 1082 connect the processors to their respective memories, namely memory 1032 and memory 1034, which may be part of the main memory locally attached to their respective processors. Although MCs 1072 and 1082 are shown as integrated with processing elements 1070 and 1080, in alternative embodiments, the MC logic may be separate logic outside of processing elements 1070 and 1080, rather than being integrated with them.

[0104] The first processing element 1070 and the second processing element 1080 may be coupled to the I / O subsystem 1090 via PP interconnects 1076 and 1086, respectively. As shown in Figure 13, the I / O subsystem 1090 includes PP interfaces 1094 and 1098. Furthermore, the I / O subsystem 1090 includes interface 1092 that couples the I / O subsystem 1090 with the high-performance graphics engine 1038. In one embodiment, bus 1049 may be used to couple the graphics engine 1038 with the I / O subsystem 1090. Alternatively, a point-to-point interconnect may couple these components.

[0105] Furthermore, the I / O subsystem 1090 may be coupled to the first bus 1016 via interface 1096. In one embodiment, the first bus 1016 may be a Peripheral Component Interconnect (PCI) bus, a PCI Express bus, or another third-generation I / O interconnect bus, but the scope of the embodiment is not limited to these.

[0106] As shown in Figure 13, various I / O devices 1014 (e.g., bioscanners, speakers, cameras, sensors) may be coupled to the first bus 1016, along with a bus bridge 1018 that can connect the first bus 1016 to the second bus 1020. In one embodiment, the second bus 1020 may be a low-pin-count (LPC) bus. In one embodiment, various devices including, for example, a keyboard / mouse 1012, a communication device 1026, and a data storage unit 1019 such as a disk drive or other mass storage device that may include code 1030 may be coupled to the second bus 1020. The illustrated code 1030 may implement one or more embodiments of, for example, the already discussed architecture 100 (Figure 1), architecture 300 (Figures 2A-2B), timing diagram 400 (Figure 4), method 500 (Figure 5), method 530 (Figure 6), structure 550 (Figure 7), method 420 (Figure 8), and / or method 440 (Figure 9). Furthermore, the audio I / O 1024 may be coupled to the second bus 1020, and the battery 1010 may supply power to the computing system 1000.

[0107] It should be noted that other embodiments are conceivable. For example, instead of the point-to-point architecture of Figure 12, the system may implement a multidrop bus or another such communication topology. Also, the elements of Figure 12 may be divided using more or fewer integrated chips than those shown in Figure 12.

[0108] Additional notes and examples

[0109] Example 1 includes a computing system comprising a system-on-a-chip having a plurality of processing engines and a memory having a set of executable program instructions, wherein, when executed by the system-on-a-chip, the instructions cause the computing system to: use a first processing engine among the plurality of processing engines to determine a first partial similarity measurement based on a first portion of a query vector and a first portion of a first candidate vector; use a second processing engine among the plurality of processing engines to determine a total similarity measurement based on the query vector and a second candidate vector; and use the first processing engine to determine whether or not to compare a second portion of the query vector with a second portion of the first candidate vector based on the first partial similarity measurement and the total similarity measurement.

[0110] Example 2 includes the computing system of Example 1, wherein, when the instruction is executed, the computing system is instructed to further use the first processing engine to compare the second portion of the query vector with the second portion of the first candidate vector in response that the first partial similarity measurement is less than the total similarity measurement, the first partial similarity measurement being the partial distance and the total similarity measurement being the total distance.

[0111] Example 3 includes the computing system described in Example 1, wherein, when the instruction is executed, the computing system further causes the computing system to use the plurality of processing engines to search for different candidate vectors, to use the plurality of processing engines to determine a plurality of partial similarity measurements between a first portion of the query vector and a first portion of the different candidate vectors, and to use the plurality of processing engines to determine, based on the plurality of partial similarity measurements and the total similarity measurement, whether or not to bypass the calculation of partial similarity between a second portion of the query vector and a second portion of the different candidate vectors.

[0112] Example 4 includes the computing system described in Example 3, wherein the system-on-chip has a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas store the different candidate vectors, and the different candidate vectors represent the vector candidate database.

[0113] Example 5 includes the computing system of Example 1, wherein, when the instruction is executed, the computing system determines, using the first processing engine, to bypass the partial similarity calculation of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, and the index for the query vector is the value at which the first partial similarity measurement is determined; in response to the bypass of the partial similarity calculation, it further determines, using the first processing engine, whether to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector, and both the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index.

[0114] Example 6 includes a computing system according to any one of Examples 1 to 5, wherein, when the instruction is executed, the computing system further causes the total similarity measurement and a plurality of similarity measurements to be stored in a maximum heap binary tree or a minimum heap binary tree, the plurality of similarity measurements being determined based on different candidate vectors and the query vector, and the total similarity measurement being greater than each of the plurality of similarity measurements.

[0115] Example 7 includes the computing system described in Example 1, wherein, when the instruction is executed, the computing system further causes the computing system to store a plurality of candidate vectors in a plurality of ping-pong buffers, to use the plurality of processing engines to determine a plurality of partial similarity measures based on a first portion of the plurality of query vectors and a first portion of the plurality of candidate vectors, and to use the plurality of processing engines to determine, based on a first subset of the plurality of partial similarity measures and a first total similarity measure, that the similarity calculation associated with a first subset of the plurality of candidate vectors will be bypassed.

[0116] Example 8 includes the computing system described in Example 7, wherein, when the instruction is executed, the computing system determines, using the group of processing engines, that a second subset of the plurality of candidate vectors will be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; uses the group of processing engines to determine a second total similarity measure based on the second subset of the plurality of candidate vectors and the plurality of query vectors; and stores each of the second total similarity measures in a different heap memory based on the identification information of one of the plurality of query vectors associated with each of the total similarity measures, each of which is dedicated to one of the plurality of query vectors.

[0117] Example 9 includes a semiconductor device comprising one or more substrates and logic coupled to the one or more substrates, wherein the logic is implemented in one or more configurable logic or fixed-function logic hardware, and the logic coupled to the one or more substrates performs the following: using a first processing engine of a plurality of processing engines to determine a first partial similarity measurement based on a first portion of a query vector and a first portion of a first candidate vector; using a second processing engine of the plurality of processing engines to determine a total similarity measurement based on the query vector and a second candidate vector; and using the first processing engine to determine whether or not to compare a second portion of the query vector with a second portion of the first candidate vector based on the first partial similarity measurement and the total similarity measurement.

[0118] Example 10 includes the apparatus described in Example 9, wherein the logic coupled to one or more substrates uses the first processing engine to compare the second portion of the query vector with the second portion of the first candidate vector in response to the first partial similarity measurement being less than the total similarity measurement, the first partial similarity measurement being the partial distance and the total similarity measurement being the total distance.

[0119] Example 11 includes the apparatus described in Example 9, wherein the logic coupled to one or more substrates uses the plurality of processing engines to search for different candidate vectors, uses the plurality of processing engines to determine a plurality of partial similarity measurements between a first portion of the query vector and a first portion of the different candidate vectors, and uses the plurality of processing engines to determine whether to bypass the partial similarity calculation between a second portion of the query vector and a second portion of the different candidate vectors based on the plurality of partial similarity measurements and the total similarity measurement.

[0120] Example 12 includes the apparatus described in Example 11, wherein the logic coupled to one or more substrates accesses a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas store the different candidate vectors, and the different candidate vectors represent a vector candidate database.

[0121] Example 13 includes the apparatus of Example 9, wherein the logic coupled to one or more substrates determines, using the first processing engine, to bypass the similarity calculation of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, that the index for the query vector is the value at which the first partial similarity measurement is determined; in response to the bypass of the similarity calculation of the first candidate vector, to use the first processing engine to increment the value of the index; and using the first processing engine to determine whether to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector, and both the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index.

[0122] Example 14 includes the apparatus described in any one of Examples 9 to 13, wherein the logic coupled to one or more substrates stores the total similarity measurement and the multiple similarity measurements in a maximum heap binary tree or a minimum heap binary tree, the multiple similarity measurements are determined based on different candidate vectors and the query vector, and the total similarity measurement is greater than each of the multiple similarity measurements.

[0123] Example 15 includes the apparatus described in Example 9, wherein the logic coupled to one or more substrates stores a plurality of candidate vectors in a plurality of ping-pong buffers, uses a plurality of processing engines to determine a plurality of partial similarity measurements based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors, and uses a plurality of processing engines to determine, based on a first subset of the plurality of partial similarity measurements and a first total similarity measurement, that a similarity calculation associated with a first subset of the plurality of candidate vectors should be bypassed.

[0124] Example 16 includes the apparatus of Example 15, wherein the logic coupled to one or more substrates determines, using a group of processing engines, that a second subset of the plurality of candidate vectors will be processed based on a second subset of the plurality of partial similarity measurements and the first total similarity measurement; uses the group of processing engines to determine a second total similarity measurement based on the second subset of the plurality of candidate vectors and the plurality of query vectors; and stores each of the second total similarity measurements in different heap memories based on the identification information of one of the plurality of query vectors associated with each of the total similarity measurements, each of which is dedicated to one of the plurality of query vectors.

[0125] Example 17 includes the apparatus described in Example 9, wherein the logic coupled to the one or more substrates includes a transistor channel region located within the one or more substrates.

[0126] Example 18 includes at least one computer-readable storage medium comprising a set of executable program instructions, which, when executed by a computing system, cause the computing system to: use a first processing engine among a plurality of processing engines to determine a first partial similarity measure based on a first portion of a query vector and a first portion of a first candidate vector; use a second processing engine among the plurality of processing engines to determine a total similarity measure based on the query vector and a second candidate vector; and use the first processing engine to determine whether to compare a second portion of the query vector with a second portion of the first candidate vector based on the first partial similarity measure and the total similarity measure.

[0127] Example 19 causes the instruction, when executed, to cause the computing system to further use the first processing engine to compare the second portion of the query vector with the second portion of the first candidate vector in response that the first partial similarity measurement is less than the total similarity measurement, and includes at least one computer-readable storage medium as in Example 18, where the first partial similarity measurement is the partial distance and the total similarity measurement is the total distance.

[0128] Example 20 includes at least one computer-readable storage medium as in Example 18, wherein, when the instruction is executed, the computing system further causes the computing system to use the plurality of processing engines to search for different candidate vectors; to use the plurality of processing engines to determine a plurality of partial similarity measurements between a first portion of the query vector and a first portion of the different candidate vectors; and to use the plurality of processing engines to determine, based on the plurality of partial similarity measurements and the total similarity measurement, whether to bypass the partial similarity calculation between a second portion of the query vector and a second portion of the different candidate vectors.

[0129] Example 21 further causes the computing system, when the instruction is executed, to access a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, wherein the plurality of memory storage areas store the different candidate vectors, and the different candidate vectors represent a vector candidate database, comprising at least one computer-readable storage medium as in Example 20.

[0130] Example 22 includes at least one computer-readable storage medium as in Example 18, wherein, when the instruction is executed, the computing system determines, using the first processing engine, to bypass the partial similarity calculation of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, that the index for the query vector is the value at which the first partial similarity measurement is determined; in response to the bypass of the partial similarity calculation, to use the first processing engine to increment the value of the index; and to use the first processing engine to determine whether to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector, wherein both the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index.

[0131] Example 23 includes at least one computer-readable storage medium as described in any one of Examples 18 to 22, wherein, when the instruction is executed, the computing system further causes the total similarity measurement and the multiple similarity measurements to be stored in a maximum heap binary tree or a minimum heap binary tree, wherein the multiple similarity measurements are determined based on different candidate vectors and the query vector, and the total similarity measurement is greater than each of the multiple similarity measurements.

[0132] Example 24 includes at least one computer-readable storage medium as in Example 18, wherein, when the instruction is executed, the computing system further causes the computing system to store a plurality of candidate vectors in a plurality of ping-pong buffers, to use the plurality of processing engines to determine a plurality of partial similarity measures based on a first portion of the plurality of query vectors and the first portion of the plurality of candidate vectors, and to use the plurality of processing engines to determine that a similarity calculation associated with a first subset of the plurality of candidate vectors should be bypassed based on a first subset of the plurality of partial similarity measures and a first total similarity measure.

[0133] Example 25 causes the computing system, when the instruction is executed, to determine using the group of processing engines that a second subset of the plurality of candidate vectors will be processed based on a second subset of the plurality of partial similarity measures and the first total similarity measure; to determine a second total similarity measure using the group of processing engines based on the second subset of the plurality of candidate vectors and the plurality of query vectors; and to store each of the second total similarity measures in a different heap memory based on the identification information of one of the plurality of query vectors associated with each of the second total similarity measures, wherein each of the different heap memories includes at least one computer-readable storage medium as in Example 24, which is dedicated to one of the plurality of query vectors.

[0134] Example 26 includes a semiconductor device comprising: means for determining a first partial similarity measurement based on a first portion of a query vector and a first portion of a first candidate vector using a first processing engine among a plurality of processing engines; means for determining a total similarity measurement based on the query vector and a second candidate vector using a second processing engine among the plurality of processing engines; and means for determining whether or not to compare a second portion of the query vector with a second portion of the first candidate vector using the first processing engine, based on the first partial similarity measurement and the total similarity measurement.

[0135] Example 27 further includes means for comparing the second portion of the query vector with the second portion of the first candidate vector using the first processing engine in response that the first partial similarity measurement is less than the total similarity measurement, wherein the first partial similarity measurement is a partial distance and the total similarity measurement is a total distance, the apparatus described in Example 26.

[0136] Example 28 includes the apparatus described in Example 26, further comprising: means for finding different candidate vectors using the plurality of processing engines; means for determining a plurality of partial similarity measurements between a first portion of the query vector and a first portion of the different candidate vectors using the plurality of processing engines; and means for determining, using the plurality of processing engines, whether or not to bypass the partial similarity calculation between a second portion of the query vector and a second portion of the different candidate vectors based on the plurality of partial similarity measurements and the total similarity measurement.

[0137] Example 29 further includes means for accessing a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, wherein the plurality of memory storage areas store the different candidate vectors, and the different candidate vectors represent a vector candidate database, as described in Example 28.

[0138] Example 30 includes the apparatus described in Example 26, comprising: means for determining, using the first processing engine, to bypass the similarity calculation of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, wherein the index for the query vector is the value at which the first partial similarity measurement is determined; means for using the first processing engine to increment the value of the index in response to the bypass of the similarity calculation of the first candidate vector; and means for using the first processing engine to determine whether or not to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector, wherein both the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index.

[0139] Example 31 further includes means for storing the total similarity measurement and a plurality of similarity measurements in a maximum heap binary tree or a minimum heap binary tree, wherein the plurality of similarity measurements are determined based on different candidate vectors and the query vector, and the total similarity measurement is greater than each of the plurality of similarity measurements, as described in any one of Examples 26 to 30.

[0140] Example 32 includes the apparatus described in Example 26, further comprising means for storing a plurality of candidate vectors in a plurality of ping-pong buffers; means for determining a plurality of partial similarity measurements based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors using the plurality of processing engines; and means for determining, based on a first subset of the plurality of partial similarity measurements and a first total similarity measurement, that a similarity calculation associated with a first subset of the plurality of candidate vectors will be bypassed using the plurality of processing engines.

[0141] Example 33 further includes the apparatus of Example 32, comprising: means for determining, using a group of processing engines, that a second subset of the plurality of candidate vectors will be processed based on a second subset of the plurality of partial similarity measurements and the first total similarity measurement; means for determining a second total similarity measurement based on the second subset of the plurality of candidate vectors and the plurality of query vectors, using the group of processing engines; and means for storing each of the second total similarity measurements in different heap memories based on identification information of one of the plurality of query vectors associated with each of the total similarity measurements, wherein each of the different heap memories is dedicated to one of the plurality of query vectors.

[0142] Thus, the techniques described herein can provide improved matching and query analysis that can efficiently retrieve results. Furthermore, queries may be batch processes to facilitate low-latency analysis. The embodiments described herein can also reduce memory footprint, latency, and processing power.

[0143] The embodiments are applicable to use with all types of semiconductor integrated circuit ("IC") chips. Examples of these IC chips include, but are not limited to, processors, controllers, chipset components, programmable logic arrays (PLAs), memory chips, network chips, system-on-a-chip (SoCs), SSD / NAND controller ASICs, etc. In addition, in some of the drawings, signal lines are represented by lines. Some may be different to indicate more component signal paths, may have numbered labels to indicate multiple component signal paths, and / or may have arrows at one or more ends to indicate the main information flow direction. However, this should not be interpreted restrictively. Rather, such additional details may be used in relation to one or more exemplary embodiments to facilitate a more easily understood circuit. Any signal line presented, whether or not it has additional information, may actually contain one or more signals that can travel in multiple directions and may be implemented in any suitable type of signaling scheme, such as digital or analog lines implemented in differential pairs, optical fiber lines, and / or single-ended lines.

[0144] While exemplary sizes / models / values / ranges may be given, embodiments are not limited thereto. It is anticipated that smaller devices may be manufactured as manufacturing techniques (e.g., photolithography) mature over time. In addition, for the sake of illustrative and descriptive brevity, and to avoid obscuring specific aspects of the embodiments, known power / ground connections to IC chips and other components may or may not be shown in the figures. Furthermore, to avoid obscuring embodiments, and given that details regarding the implementation of such block diagram configurations depend heavily on the platform on which the embodiment is to be implemented, i.e., such details are well known to those skilled in the art, the configurations may be shown in block diagram form. Where specific details (e.g., circuits) are described to illustrate exemplary embodiments, it will be apparent to those skilled in the art that embodiments can be carried out without these specific details, or with modifications of these specific details. Therefore, the description should be considered illustrative, not restrictive.

[0145] The term “connection” may be used herein to refer to any type of direct or indirect relationship between the components in question, and may apply to electrical, mechanical, fluid, optical, electromagnetic, electromechanical, or other connections. In addition, terms such as “first,” “second,” etc., may be used herein solely for the purpose of facilitating the discussion and, unless otherwise indicated, do not have any specific temporal or chronological meaning.

[0146] When used in the present application and claims, the list of items joined by the term “one or more of” may mean any combination of the enumerated terms. For example, the phrase “one or more of A, B, or C” may mean A, B, C, A and B, A and C, B and C, or A, B and C.

[0147] Those skilled in the art will understand from the foregoing description that the broad techniques of the embodiments can be implemented in a variety of forms. Therefore, although the embodiments have been described in relation to those specific examples, the true scope of the embodiments should not be limited thereto, as other modifications will become apparent to those skilled in the art upon examination of the drawings, specification, and the following claims. [Other adjacent items] [Item 1] A system-on-a-chip with multiple processing engines, A memory containing a set of executable program instructions A computing system comprising, where the instruction, when executed by the system-on-chip, the computing system Using the first processing engine among the plurality of processing engines, a first partial similarity measurement is determined based on the first part of the query vector and the first part of the first candidate vector, Using the second processing engine among the plurality of processing engines, the total similarity measurement value is determined based on the query vector and the second candidate vector. Using the first processing engine, determine whether or not to compare the second portion of the query vector with the second portion of the first candidate vector based on the first partial similarity measurement and the total similarity measurement. A computing system that performs this task. [Item 2] When the instruction is executed, the computing system will: The computing system according to item 1, wherein, in response to the first partial similarity measurement being less than the total similarity measurement, the first processing engine is used to further compare the second portion of the query vector with the second portion of the first candidate vector, the first partial similarity measurement being the partial distance and the total similarity measurement being the total distance. [Item 3] When the instruction is executed, the computing system will: Using the aforementioned multiple processing engines, different candidate vectors are searched for, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined between the first portion of the query vector and the first portion of the different candidate vectors. Using the aforementioned multiple processing engines, a determination is made, based on the multiple partial similarity measurements and the total similarity measurements, whether or not to bypass the calculation of partial similarity between the second part of the query vector and the second part of the different candidate vectors. A computing system as described in item 1, which further performs the above. [Item 4] The computing system according to item 3, wherein the system-on-chip has a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas store the different candidate vectors, and the different candidate vectors represent a vector candidate database. [Item 5] When the instruction is executed, the computing system will: The first processing engine determines to bypass the calculation of partial similarity of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, wherein the index for the query vector is the value at which the first partial similarity measurement is determined. In response to the bypass of the partial similarity calculation, the first processing engine is used to increment the value of the index, Using the first processing engine, a decision is made as to whether or not to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector, The computing system according to item 1, wherein the feature values ​​of the third candidate vector and the feature values ​​of the query vector are both associated with the incremented value of the index. [Item 6] When the instruction is executed, the computing system will: The computing system according to item 1, further comprising storing the total similarity measurement and a plurality of similarity measurements in a maximum heap binary tree or a minimum heap binary tree, wherein the plurality of similarity measurements are determined based on different candidate vectors and the query vector, and the total similarity measurement is greater than each of the plurality of similarity measurements. [Item 7] When the instruction is executed, the computing system will: Storing multiple candidate vectors in multiple ping-pong buffers, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors. Using the aforementioned multiple processing engines, it is determined that the similarity calculation associated with the first subset of the multiple candidate vectors will be bypassed, based on the first subset of the multiple partial similarity measurements and the first total similarity measurement. A computing system as described in item 1, which further performs the above. [Item 8] When the instruction is executed, the computing system will: Using the group of processing engines, it is determined that a second subset of the plurality of candidate vectors will be processed based on the second subset of the plurality of partial similarity measurements and the first total similarity measurement, Using the group of the plurality of processing engines, a second total similarity measurement is determined based on the second subset of the plurality of candidate vectors and the plurality of query vectors, Each of the two total similarity measurements is stored in a different heap memory based on the identification information of one of the multiple query vectors associated with each of the two total similarity measurements. The computing system described in item 7, further comprising the different heap memories, each of which is dedicated to one of the multiple query vectors. [Item 9] One or more circuit boards, The logic coupled to one or more of the aforementioned boards A semiconductor device comprising, wherein the logic is implemented in one or more of configurable logic or fixed-function logic hardware, and the logic coupled to the one or more substrates is, Using the first processing engine among multiple processing engines, a first partial similarity measure is determined based on the first part of the query vector and the first part of the first candidate vector, Using the second processing engine among the plurality of processing engines, the total similarity measurement value is determined based on the query vector and the second candidate vector. Using the first processing engine, determine whether or not to compare the second portion of the query vector with the second portion of the first candidate vector based on the first partial similarity measurement and the total similarity measurement. A semiconductor device that performs this function. [Item 10] The logic coupled to the one or more substrates is The apparatus according to item 9, wherein, in response to the first partial similarity measurement being less than the total similarity measurement, the first processing engine compares the second portion of the query vector with the second portion of the first candidate vector, the first partial similarity measurement being the partial distance and the total similarity measurement being the total distance. [Item 11] The logic coupled to the one or more substrates is Using the aforementioned multiple processing engines, different candidate vectors are searched for, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined between the first portion of the query vector and the first portion of the different candidate vectors. Using the aforementioned multiple processing engines, a determination is made, based on the multiple partial similarity measurements and the total similarity measurements, whether or not to bypass the calculation of partial similarity between the second part of the query vector and the second part of the different candidate vectors. The apparatus described in item 9, which performs the following actions. [Item 12] The apparatus according to item 11, wherein the logic coupled to one or more substrates accesses a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas store the different candidate vectors, and the different candidate vectors represent a vector candidate database. [Item 13] The logic coupled to the one or more substrates is The first processing engine determines to bypass the similarity calculation of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, wherein the index for the query vector is the value at which the first partial similarity measurement is determined. In response to the bypass of the similarity calculation of the first candidate vector, the first processing engine is used to increment the value of the index, Using the first processing engine, a decision is made as to whether or not to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector, The apparatus according to item 9, wherein both the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index. [Item 14] The logic coupled to the one or more substrates is The apparatus according to item 9, wherein the total similarity measurement and a plurality of similarity measurements are stored in a maximum heap binary tree or a minimum heap binary tree, the plurality of similarity measurements are determined based on different candidate vectors and the query vector, and the total similarity measurement is greater than each of the plurality of similarity measurements. [Item 15] The logic coupled to the one or more substrates is Storing multiple candidate vectors in multiple ping-pong buffers, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors. Using the aforementioned multiple processing engines, it is determined that the similarity calculation associated with the first subset of the multiple candidate vectors will be bypassed, based on the first subset of the multiple partial similarity measurements and the first total similarity measurement. The apparatus described in item 9, which performs the following actions. [Item 16] The logic coupled to the one or more substrates is Using the group of processing engines, it is determined that a second subset of the plurality of candidate vectors will be processed based on the second subset of the plurality of partial similarity measurements and the first total similarity measurement, Using the group of the plurality of processing engines, a second total similarity measurement is determined based on the second subset of the plurality of candidate vectors and the plurality of query vectors, Each of the two total similarity measurements is stored in a different heap memory based on the identification information of one of the multiple query vectors associated with each of the two total similarity measurements. The apparatus described in item 15, wherein each of the different heap memories is dedicated to one of the plurality of query vectors. [Item 17] The apparatus according to item 9, wherein the logic coupled to the one or more substrates includes transistor channel regions positioned within the one or more substrates. [Item 18] A computer-readable storage medium comprising a set of executable program instructions, wherein, when the instructions are executed by a computing system, the computing system... Using the first processing engine among multiple processing engines, a first partial similarity measure is determined based on the first part of the query vector and the first part of the first candidate vector, Using the second processing engine among the plurality of processing engines, the total similarity measurement value is determined based on the query vector and the second candidate vector. Using the first processing engine, determine whether or not to compare the second portion of the query vector with the second portion of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, A computer-readable storage medium that enables the following. [Item 19] When the instruction is executed, the computing system will: At least one computer-readable storage medium as described in item 18, in response to the first partial similarity measurement being less than the total similarity measurement, the first processing engine is used to further compare the second portion of the query vector with the second portion of the first candidate vector, wherein the first partial similarity measurement is a partial distance and the total similarity measurement is a total distance. [Item 20] When the instruction is executed, the computing system will: Using the aforementioned multiple processing engines, different candidate vectors are searched for, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined between the first portion of the query vector and the first portion of the different candidate vectors. Using the aforementioned multiple processing engines, a determination is made, based on the multiple partial similarity measurements and the total similarity measurements, whether or not to bypass the calculation of partial similarity between the second part of the query vector and the second part of the different candidate vectors. Further, at least one computer-readable storage medium as described in item 18. [Item 21] When the instruction is executed, the computing system will: A computer-readable storage medium as described in item 20, wherein one of the processing engines is further configured to access a plurality of memory storage areas, each dedicated to it, the plurality of memory storage areas storing the different candidate vectors, and the different candidate vectors represent the vector candidate database. [Item 22] When the instruction is executed, the computing system will: The first processing engine determines to bypass the calculation of partial similarity of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, wherein the index for the query vector is the value at which the first partial similarity measurement is determined. In response to the bypass of the partial similarity calculation, the first processing engine is used to increment the value of the index, Using the first processing engine, a decision is made as to whether or not to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector, Further, the feature values ​​of the third candidate vector and the feature values ​​of the query vector are associated with the incremented value of the index in at least one computer-readable storage medium as described in item 18. [Item 23] When the instruction is executed, the computing system will: The computer-readable storage medium described in item 18 further stores the total similarity measurement and the plurality of similarity measurements in a maximum heap binary tree or a minimum heap binary tree, wherein the plurality of similarity measurements are determined based on different candidate vectors and the query vector, and the total similarity measurement is greater than each of the plurality of similarity measurements. [Item 24] When the instruction is executed, the computing system will: Storing multiple candidate vectors in multiple ping-pong buffers, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors. Using the aforementioned multiple processing engines, it is determined that the similarity calculation associated with the first subset of the multiple candidate vectors will be bypassed based on the first subset of the multiple partial similarity measurements and the first total similarity measurement. Further, at least one computer-readable storage medium as described in item 18. [Item 25] When the instruction is executed, the computing system will: Using the group of processing engines, it is determined that a second subset of the plurality of candidate vectors will be processed based on the second subset of the plurality of partial similarity measurements and the first total similarity measurement, Using the group of the plurality of processing engines, a second total similarity measurement is determined based on the second subset of the plurality of candidate vectors and the plurality of query vectors, Each of the two total similarity measurements is stored in a different heap memory based on the identification information of one of the multiple query vectors associated with each of the two total similarity measurements. Further, the computer-readable storage medium described in item 24, wherein each of the different heap memories is dedicated to one of the plurality of query vectors.

Claims

1. A system-on-a-chip with multiple processing engines, A memory containing a set of executable program instructions A computing system comprising, where the executable program instruction, when executed by the system-on-chip, Using the first processing engine among the plurality of processing engines, a first partial similarity measurement is determined based on the first part of the query vector and the first part of the first candidate vector. Using the second processing engine among the plurality of processing engines, the total similarity measurement value is determined based on the query vector and the second candidate vector. Using the first processing engine, determine whether or not to compare the second portion of the query vector with the second portion of the first candidate vector based on the first partial similarity measurement and the total similarity measurement. A computing system that performs this task.

2. When the aforementioned executable program instruction is executed, it will cause the computing system to: The computing system according to claim 1, wherein, in response to the first partial similarity measurement being less than the total similarity measurement, the first processing engine is used to further compare the second portion of the query vector with the second portion of the first candidate vector, the first partial similarity measurement being a partial distance and the total similarity measurement being a total distance.

3. When the aforementioned executable program instruction is executed, it will cause the computing system to: Using the aforementioned multiple processing engines, different candidate vectors are searched for, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined between the first portion of the query vector and the first portion of the different candidate vectors. Using the aforementioned multiple processing engines, a determination is made, based on the multiple partial similarity measurements and the total similarity measurements, whether or not to bypass the calculation of partial similarity between the second part of the query vector and the second part of the different candidate vectors. The computing system according to claim 1 or 2, further comprising the following:

4. The computing system according to claim 3, wherein the system-on-chip has a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas store the different candidate vectors, and the different candidate vectors represent a vector candidate database.

5. When the aforementioned executable program instruction is executed, it will cause the computing system to: The first processing engine determines to bypass the calculation of partial similarity of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, wherein the index for the query vector is the value at which the first partial similarity measurement is determined. In response to the bypass of the partial similarity calculation, the first processing engine is used to increment the value of the index, Using the first processing engine, a decision is made as to whether or not to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector. The computing system according to claim 1, wherein the feature values ​​of the third candidate vector and the feature values ​​of the query vector are both associated with the incremented value of the index.

6. When the aforementioned executable program instruction is executed, it will cause the computing system to: The computing system according to any one of claims 1 to 5, further comprising storing the total similarity measurement and a plurality of similarity measurements in a maximum heap binary tree or a minimum heap binary tree, wherein the plurality of similarity measurements are determined based on different candidate vectors and the query vector, and the total similarity measurement is greater than each of the plurality of similarity measurements.

7. When the aforementioned executable program instruction is executed, it will cause the computing system to: Storing multiple candidate vectors in multiple ping-pong buffers, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors. Using the aforementioned multiple processing engines, it is determined that the similarity calculation associated with the first subset of the multiple candidate vectors will be bypassed, based on the first subset of the multiple partial similarity measurements and the first total similarity measurement. The computing system according to claim 1, further comprising the following:

8. When the aforementioned executable program instruction is executed, it will cause the computing system to: Using the group of processing engines described above, it is determined that a second subset of the plurality of candidate vectors will be processed based on a second subset of the plurality of partial similarity measurements and the first total similarity measurement, Using the group of the plurality of processing engines, a second total similarity measurement is determined based on the second subset of the plurality of candidate vectors and the plurality of query vectors, Each of the two total similarity measurements is stored in a different heap memory based on the identification information of one of the multiple query vectors associated with each of the two total similarity measurements. The computing system according to claim 7, further comprising the following steps, wherein each of the different heap memories is dedicated to one of the plurality of query vectors.

9. One or more circuit boards, The logic coupled to one or more of the aforementioned boards A semiconductor device comprising, wherein the logic is implemented in one or more of configurable logic or fixed-function logic hardware, and the logic coupled to one or more substrates is, Using a first processing engine among multiple processing engines, a first partial similarity measurement is determined based on a first part of the query vector and a first part of the first candidate vector. Using the second processing engine among the plurality of processing engines, the total similarity measurement value is determined based on the query vector and the second candidate vector. Using the first processing engine, determine whether or not to compare the second portion of the query vector with the second portion of the first candidate vector based on the first partial similarity measurement and the total similarity measurement. A semiconductor device that performs this function.

10. The logic coupled to the one or more substrates is The semiconductor device according to claim 9, wherein, using the first processing engine, in response to the first partial similarity measurement being less than the total similarity measurement, the second portion of the query vector is compared with the second portion of the first candidate vector, the first partial similarity measurement is the partial distance, and the total similarity measurement is the total distance.

11. The logic coupled to the one or more substrates is Using the aforementioned multiple processing engines, different candidate vectors are searched for, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined between the first portion of the query vector and the first portion of the different candidate vectors. Using the aforementioned multiple processing engines, a determination is made, based on the multiple partial similarity measurements and the total similarity measurements, whether or not to bypass the calculation of partial similarity between the second part of the query vector and the second part of the different candidate vectors. A semiconductor device according to claim 9 or 10, which performs the following:

12. The semiconductor device according to claim 11, wherein the logic coupled to one or more substrates accesses a plurality of memory storage areas, each dedicated to one of the plurality of processing engines, the plurality of memory storage areas store the different candidate vectors, and the different candidate vectors represent a vector candidate database.

13. The logic coupled to the one or more substrates is The first processing engine determines to bypass the similarity calculation of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, wherein the index for the query vector is the value at which the first partial similarity measurement is determined. In response to the bypass of the similarity calculation of the first candidate vector, the first processing engine is used to increment the value of the index, Using the first processing engine, a decision is made as to whether or not to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector, The semiconductor device according to claim 9, wherein both the feature value of the third candidate vector and the feature value of the query vector are associated with the incremented value of the index.

14. The logic coupled to the one or more substrates is The semiconductor device according to any one of claims 9 to 13, wherein the total similarity measurement value and a plurality of similarity measurement values ​​are stored in a maximum heap binary tree or a minimum heap binary tree, the plurality of similarity measurement values ​​are determined based on different candidate vectors and the query vector, and the total similarity measurement value is greater than each of the plurality of similarity measurement values.

15. The logic coupled to the one or more substrates is Storing multiple candidate vectors in multiple ping-pong buffers, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors. Using the aforementioned multiple processing engines, it is determined that the similarity calculation associated with the first subset of the multiple candidate vectors will be bypassed, based on the first subset of the multiple partial similarity measurements and the first total similarity measurement. A semiconductor device according to claim 9, which performs the following:

16. The logic coupled to the one or more substrates is Using the group of processing engines described above, it is determined that a second subset of the plurality of candidate vectors will be processed based on a second subset of the plurality of partial similarity measurements and the first total similarity measurement, Using the group of the plurality of processing engines, a second total similarity measurement is determined based on the second subset of the plurality of candidate vectors and the plurality of query vectors, Each of the two total similarity measurements is stored in a different heap memory based on the identification information of one of the multiple query vectors associated with each of the two total similarity measurements. The semiconductor device according to claim 15, wherein each of the different heap memories is dedicated to one of the plurality of query vectors.

17. The semiconductor device according to claim 9, wherein the logic coupled to the one or more substrates includes a transistor channel region positioned within the one or more substrates.

18. A computer program comprising a set of executable instructions, wherein, when the instructions are executed by a computing system, the computing system... Using a first processing engine among multiple processing engines, a first partial similarity measurement is determined based on a first part of the query vector and a first part of the first candidate vector. Using the second processing engine among the plurality of processing engines, the total similarity measurement value is determined based on the query vector and the second candidate vector. Using the first processing engine, determine whether or not to compare the second portion of the query vector with the second portion of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, A computer program that performs an action.

19. When the instruction is executed, the computing system will: The computer program according to claim 18, wherein, in response to the first partial similarity measurement being less than the total similarity measurement, the first processing engine is used to further compare the second portion of the query vector with the second portion of the first candidate vector, the first partial similarity measurement being the partial distance and the total similarity measurement being the total distance.

20. When the instruction is executed, the computing system will: Using the aforementioned multiple processing engines, different candidate vectors are searched for, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined between the first portion of the query vector and the first portion of the different candidate vectors. Using the aforementioned multiple processing engines, a determination is made, based on the multiple partial similarity measurements and the total similarity measurements, whether or not to bypass the calculation of partial similarity between the second part of the query vector and the second part of the different candidate vectors. A computer program according to claim 18 or 19, which further causes the computer program to perform the following.

21. When the instruction is executed, the computing system will: The computer program according to claim 20, wherein one of the plurality of processing engines is further made to access a plurality of memory storage areas, each dedicated to it, the plurality of memory storage areas storing the different candidate vectors, and the different candidate vectors represent a vector candidate database.

22. When the instruction is executed, the computing system will: The first processing engine determines to bypass the calculation of partial similarity of the first candidate vector based on the first partial similarity measurement and the total similarity measurement, wherein the index for the query vector is the value at which the first partial similarity measurement is determined. In response to the bypass of the partial similarity calculation, the first processing engine is used to increment the value of the index, Using the first processing engine, a decision is made as to whether or not to bypass the similarity calculation of the third candidate vector based on a partial similarity measurement that will be determined based on the feature values ​​of the third candidate vector and the feature values ​​of the query vector, The computer program according to claim 18, further comprising the following steps, wherein the feature values ​​of the third candidate vector and the feature values ​​of the query vector are both associated with the incremented value of the index.

23. When the instruction is executed, the computing system will: The computer program according to any one of claims 18 to 22, further comprising storing the total similarity measurement and a plurality of similarity measurements in a maximum heap binary tree or a minimum heap binary tree, wherein the plurality of similarity measurements are determined based on different candidate vectors and the query vector, and the total similarity measurement is greater than each of the plurality of similarity measurements.

24. When the instruction is executed, the computing system will: Storing multiple candidate vectors in multiple ping-pong buffers, Using the aforementioned multiple processing engines, a plurality of partial similarity measurements are determined based on a first portion of a plurality of query vectors and a first portion of the plurality of candidate vectors. Using the aforementioned multiple processing engines, it is determined that the similarity calculation associated with the first subset of the multiple candidate vectors will be bypassed based on the first subset of the multiple partial similarity measurements and the first total similarity measurement. The computer program according to claim 18, which further causes the computer program to perform the following.

25. When the instruction is executed, the computing system will: Using the group of processing engines described above, it is determined that a second subset of the plurality of candidate vectors will be processed based on a second subset of the plurality of partial similarity measurements and the first total similarity measurement, Using the group of the plurality of processing engines, a second total similarity measurement is determined based on the second subset of the plurality of candidate vectors and the plurality of query vectors, Each of the two total similarity measurements is stored in a different heap memory based on the identification information of one of the multiple query vectors associated with each of the two total similarity measurements. The computer program according to claim 24, further comprising the following steps, wherein each of the different heap memories is dedicated to one of the plurality of query vectors.

26. At least one computer-readable storage medium for storing a computer program according to any one of claims 18 to 25.

Images