Search device and search method

Abstract

A retrieval device and a retrieval method. The retrieval device (10) comprises a first processor (110) and a second processor (120), wherein the first processor (110) is a general central processor, used to perform table lookup operation in the retrieval process; the second processor (120) is a neural network processor, used to perform matrix / vector operation in the retrieval process. In the retrieval method, the steps of the retrieval algorithm (retrieval process) are reasonably allocated to different processors for execution, so as to exert the advantages of different kinds of processors and improve the retrieval efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of retrieval, and more particularly, to a retrieval device and retrieval method for retrieval. BACKGROUND

[0002] Retrieval refers to starting from a user's specific information needs, using certain methods, technical means, and according to certain clues and rules to find relevant information from a specific information set. Retrieval has been applied to various industries in today's society, for example, face recognition, license plate detection, or fingerprint recognition tasks can all apply various retrieval technologies.

[0003] For example, in the security field, when a target person needs to be retrieved or tracked, the face, body, and other features of the target person are usually extracted, and then these extracted features are compared with the features in the base library as retrieval features to find some features with higher similarity in order to further obtain the activity track and time information of the target person. However, such a base library is often huge, for example, a city generates about 100 billion base library data every quarter, and retrieval often needs to retrieve the target person in a very short time, for example, the retrieval result needs to be obtained within a few seconds. If the retrieval features are compared with the features in the base library one by one, the computational complexity is proportional to the number of base libraries and feature dimensions, and the time consumption will be very long in the case of such a large amount of data in the base library.

[0004] To solve the above problems, some retrieval algorithms are usually used in the prior art to perform retrieval, which is equivalent to sacrificing part of the retrieval accuracy to achieve higher retrieval performance. With the development of artificial intelligence (AI), retrieval algorithms based on artificial intelligence technology have also been applied. Such retrieval algorithms can be operated with the help of processors, the most common of which are general-purpose central processing units (CPUs) and neural-network processing units (NPUs). However, if a CPU is used, the operation speed of the CPU for complex vector calculations is slow, resulting in low retrieval efficiency. If an NPU is used, the NPU is slow in executing lookup table operations when there is no special instruction, resulting in low retrieval efficiency.

[0005] Therefore, how to improve retrieval efficiency is a technical problem to be solved. SUMMARY

[0006] The present application provides a retrieval device and retrieval method for retrieval, which can improve retrieval efficiency.

[0007] In a first aspect, a retrieval device is provided, which includes a first processor and a second processor. The first processor is a general central processing unit, configured to perform table lookup operation in a retrieval process. The second processor is a neural network processing unit, configured to perform matrix / vector operation in the retrieval process.

[0008] In the technical solution of the present application, the steps of the retrieval algorithm (retrieval process) are reasonably distributed to different processors for execution, so as to give full play to the advantages of different types of processors and improve the retrieval efficiency.

[0009] It should be noted that in the embodiments of the present application, the retrieval process includes multiple steps, which can be divided into query operation, matrix / vector operation and other operations. The other operations may, for example, include sorting operation, loading distance table, accessing distance table, selection operation, etc. Among them, the query operation is relatively flexible and needs to be executed multiple times, and the NPU is not suitable for executing such operation, while the CPU is relatively suitable, so the query operation is distributed to the first processor (CPU) for execution. The matrix / vector operation is the most suitable operation for the NPU, while the CPU is relatively unable to undertake complex matrix and vector operations, so the matrix / vector operation is distributed to the second processor (NPU) for execution. As for other operations, either the first processor or the second processor can execute them, for example, the idle processor in the two can be selected to execute, for example, the processor that undertakes fewer steps can be selected to execute, and for example, the same processor as the previous step can be selected to execute.

[0010] In combination with the first aspect, in some implementations of the first aspect, the table lookup operation specifically includes: querying distance values between a to-be-retrieved feature and multiple base features in a base library, and generating a distance matrix of the to-be-retrieved feature and the multiple base features. The to-be-retrieved feature is used to represent a feature of to-be-retrieved data, and the base feature is a feature of data in the base library. The matrix / vector operation specifically includes: accumulating the distance matrix to obtain a first distance between the to-be-retrieved feature and each base feature in the multiple base features.

[0011] In combination with the first aspect, in some implementations of the first aspect, the first processor is further configured to set the format of the distance matrix to be consistent with the format required by the second processor.

[0012] In combination with the first aspect, in some implementations of the first aspect, the first processor or the second processor is further configured to: according to the first distance, select M base features with the smallest first distance as a retrieval result, where M is a positive integer.

[0013] Optionally, in order to improve the retrieval efficiency, the base features in the base library can be clustered to obtain a plurality of cluster centers, and in the retrieval, the cluster centers with similar distances can be obtained first, and then the base features with similar distances can be further retrieved from the cluster centers with similar distances. This is equivalent to a hierarchical retrieval method, which can be understood as grouping the base features (forming a set), so that similar groups can be found first, and then similar base feature vectors can be found from the similar groups. Compared with the traditional one-to-one comparison with each base feature, the retrieval efficiency is higher. That is, the base library can include a plurality of cluster centers, and each cluster center includes a plurality of base features.

[0014] It should be noted that in the case where the base library includes a plurality of cluster centers, the second processor can be used to calculate the distance between the to-be-retrieved feature and each cluster center, for example, the second distance (matrix / vector operation) in the following, then the distance is sorted (sorting operation), and some cluster centers are selected (selection operation), and then the first processor performs the query operation, at this time, the first processor only needs to query the base features in the selected cluster centers. That is, in this process, matrix / vector operation, sorting operation and selection operation are also involved, the execution subject of these steps can refer to the above, and will not be introduced again. The only difference is that the first processor queries which base features when performing the query operation.

[0015] In combination with the first aspect, in some implementations of the first aspect, the base library further includes a plurality of cluster centers, each cluster center in the plurality of cluster centers includes a plurality of base features, and the second processor is further configured to calculate a second distance between the to-be-retrieved feature and the plurality of cluster centers; the first processor or the second processor is further configured to select K cluster centers with the smallest second distance from the plurality of cluster centers, K being a positive integer; and the first processor is specifically configured to query distance values between the to-be-retrieved feature and the plurality of base features in the K cluster centers in the form of a distance matrix.

[0016] Optionally, the accumulation of the distance matrix can be implemented by using vector operation, and the accumulation of the distance matrix can also be implemented by using matrix operation.

[0017] Optionally, when using matrix operation, the distance can be obtained by multiplying the distance matrix with a weight matrix, which can be set according to the base library features. In this case, the operation efficiency can be improved by cooperation between the first processor and the second processor. For example, when the first processor generates the distance matrix, the distance matrix can be set to be consistent with the format required by the second processor, so that the second processor can directly operate the distance matrix, thereby saving processing steps. Since matrix operation is used, the vector unit is released, and the vector unit can be used to perform other operations of the retrieval process, thereby realizing parallel algorithm and improving operation efficiency.

[0018] In combination with the first aspect, in some implementations of the first aspect, the matrix / vector operation specifically includes multiplying the distance matrix with a weight matrix to obtain the distance, and the weight matrix is set according to the base library features.

[0019] Optionally, the CPU needs to execute load and store instructions multiple times when performing the table lookup operation. In order to reduce the instruction delay of write-after-read and reduce the number of execution of the load and store instructions, the distance table can be preloaded, and the preloaded distance table can be resident in the L2 cache of the CPU. That is, the distance table needs to be accessed repeatedly when querying, so the distance table is more convenient for querying by preloading and residency. The distance table stores the distance values.

[0020] The second aspect provides a retrieval method, which includes: using a first processor to perform a table lookup operation in a retrieval process, the first processor being a CPU; and using a second processor to perform a matrix / vector operation in the retrieval process, the second processor being an NPU.

[0021] In combination with the second aspect, in some implementations of the second aspect, the table lookup operation specifically includes querying distance values between a to-be-retrieved feature and a plurality of base library features in a base library, and generating a distance matrix of the to-be-retrieved feature and the plurality of base library features, the to-be-retrieved feature being used to represent a feature of to-be-retrieved data, and the base library feature being a feature of data in the base library; and the matrix / vector operation specifically includes accumulating the distance matrix to obtain a first distance between the to-be-retrieved feature and each base library feature in the plurality of base library features.

[0022] In combination with the second aspect, in some implementations of the second aspect, the method further includes: the first processor setting a format of the distance matrix to be consistent with a format required by the second processor.

[0023] With reference to the second aspect, in some implementations of the second aspect, the method further includes: selecting, by the first processor or the second processor, M library features with the smallest first distances as the retrieval result according to the first distances, M being a positive integer.

[0024] With reference to the second aspect, in some implementations of the second aspect, the library further includes a plurality of cluster centers, each of the plurality of cluster centers including a plurality of library features, and the method further includes:

[0025] calculating, by the second processor, second distances between the feature to be retrieved and the plurality of cluster centers;

[0026] selecting, by the first processor or the second processor, K cluster centers with the smallest second distances from the plurality of cluster centers, K being a positive integer;

[0027] querying, by the first processor, distance values between the feature to be retrieved and the plurality of library features in the K cluster centers, to obtain a distance matrix.

[0028] With reference to the second aspect, in some implementations of the second aspect, the matrix / vector operation in the retrieval process includes: multiplying the distance matrix by a weight matrix to obtain the first distances, the weight matrix being set according to the library features.

[0029] In a third aspect, the present application provides a retrieval device, which includes: a processor coupled with a memory; the memory is configured to store instructions; and the processor is configured to execute the instructions stored in the memory, so that the device performs the steps of the method in the second aspect or any possible implementation manner thereof.

[0030] In a fourth aspect, the present application provides a computer readable medium storing instructions for execution by an apparatus, the instructions being configured to implement the method in the second aspect or any possible implementation manner thereof.

[0031] In a fifth aspect, the present application provides a computer program product containing instructions which, when the computer program product is run on a computer, cause the computer to execute the method in the second aspect or any possible implementation manner thereof. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is a schematic structural diagram of a retrieval system according to an embodiment of the present application.

[0033] Figure 2 is a schematic structural diagram of a retrieval device according to an embodiment of the present application.

[0034] Figure 3 is a schematic flowchart of a retrieval method according to an embodiment of the present application.

[0035] Figure 4 is a schematic flowchart of a retrieval method of an embodiment of the present application.

[0036] Figure 5 is a schematic structural diagram of a retrieval device of an embodiment of the present application.

[0037] Figure 6 is a schematic structural diagram of a retrieval device of an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0039] For the convenience of understanding, some professional terms will be introduced first.

[0040] (1) Retrieval algorithm

[0041] The retrieval algorithm can be understood as a method of retrieving data related to the to-be-retrieved data from a large data storage library by using a small amount of to-be-retrieved data. For example, in a common search engine, some links related to the keyword can be searched out by inputting the keyword, at this time, the input keyword can be regarded as to-be-retrieved data, and the related links can be regarded as data related to the to-be-retrieved data. Similarly, image retrieval can be regarded as retrieving images related to a to-be-retrieved image from an image library storing a large number of images by using a small amount of to-be-retrieved images (for example, one to-be-retrieved image).

[0042] It should be noted that the above data can be text data, image data, and other types of data such as voice and sequence in addition to keywords.

[0043] The retrieval algorithm includes various types of retrieval algorithms such as tree-based methods, hash, vector quantization methods, etc. The tree-based method (for example, KD tree) has relatively high retrieval performance when the spatial dimension is relatively low, but when the spatial dimension is relatively high, the method degenerates into brute-force enumeration, resulting in poor retrieval performance. The hash method has relatively good performance for small and medium-sized data sets, but for large data sets, it still cannot achieve good retrieval performance. The vector quantization type retrieval algorithm can quantize the features of the data, and then can filter out data with higher relevance or higher similarity by comparing the distance between the feature vectors.

[0044] (2) Product quantization (PQ) algorithm

[0045] The PQ algorithm is a common retrieval algorithm of the vector quantization type. The PQ algorithm first decomposes a high-dimensional vector space into a Cartesian product of a plurality of low-dimensional vector spaces, and quantizes each low-dimensional vector space. In this way, each vector can be represented by a combination of quantization codes of the low-dimensional vector spaces. For example, for N 128-dimensional vectors, N is a positive integer, each vector can be divided into 4 32-dimensional vectors, and 256 cluster centers are obtained by using a clustering algorithm. Each segment of the original vector is replaced by the code of the cluster center, and the code of the 256 cluster centers is represented by 1 byte. In this way, each 128-dimensional floating point vector can be represented by 4 bytes. When performing retrieval, the query vector (feature vector of the data to be retrieved) is also grouped in the same way, the distance between the query vector and each cluster center is calculated, and a 256*4 distance table is obtained. Traverse the N vectors in the base library, find the corresponding distance in the distance table according to the code, and add the 4 distances of each vector to obtain the distance between the vector and the query vector. Return the K vectors with the smallest distance as the query result. As can be seen, the PQ algorithm converts the distance calculation of a large number of high-dimensional floating point vectors into a table lookup operation of low-dimensional vectors, greatly reducing the calculation amount. On the basis of the PQ algorithm, an inverted index algorithm is added, and only part of the base library vectors are retrieved, which can further reduce the query time.

[0046] From the execution process of the PQ algorithm, the feature retrieval algorithm based on PQ includes both distance calculation of high-dimensional floating point vectors and table lookup operation with high flexibility. The PQ algorithm can run on processors or chips such as CPU, GPU, NPU or FPGA, but these processors or chips have their own shortcomings, for example, the CPU has poor vector calculation performance; the GPU has high power consumption; if there is no special instruction, the NPU has very limited performance for table lookup operation, and table lookup is a key performance bottleneck of the algorithm; and the FPGA has high cost. Therefore, it is difficult to obtain ideal retrieval performance by using any processor for retrieval operation. In view of the above technical problems, the present application provides a new retrieval method, which reasonably allocates the steps of the retrieval algorithm to multiple processors for execution, so as to take advantage of different types of processors and improve retrieval efficiency.

[0047] Figure 1 is a schematic structural diagram of the retrieval system of the embodiment of the present application. As shown in Figure 1 The retrieval system 100 can include a retrieval device 10 and a storage module 130, wherein the retrieval device 10 includes a first processor 110 and a second processor 120. In some implementations, the retrieval device 10 can be a chip.

[0048] The first processor 110 is configured to perform the table lookup operation in the retrieval algorithm. The first processor 110 can be a CPU.

[0049] Optionally, the distance table and the feature to be searched can be input to the first processor 110, the first processor 110 queries the distance table to obtain distance values between the feature to be searched and the features in the database, generates a distance matrix according to the distance values, and outputs the distance matrix to the second processor 120 or the storage module 130, so that the second processor 120 processes the distance matrix.

[0050] The second processor 120 is configured to perform matrix / vector operations in the search algorithm. The second processor 120 can be an NPU.

[0051] Optionally, the second processor 120 can receive the distance matrix of the first processor 110 or read the distance matrix from the storage module 130, and then perform matrix operations or vector operations on the distance matrix to obtain distances between the feature to be searched and the features in the database (e.g., the first distance below).

[0052] Optionally, the second processor 120 can output the distances between the feature to be searched and the features in the database to the first processor 110, so that the first processor 110 can sort the distances (i.e., perform a sorting operation), thereby selecting some of the features in the database (i.e., a selection operation).

[0053] Optionally, the second processor 120 can also output the distances between the feature to be searched and the features in the database to the storage module 130, and then the first processor 110 can read the distances from the storage module 130 and sort the distances (i.e., perform a sorting operation), thereby selecting some of the features in the database (i.e., a selection operation).

[0054] Optionally, the second processor 120 can also sort the distances between the feature to be searched and the features in the database (i.e., perform a sorting operation), and then output the sorted distances to the first processor 110 or the storage module 130, or continue to perform a selection operation after the sorting operation and then output the selected features in the database. In this case, the data interaction between the first processor 110 or the storage module 130 is omitted, which can speed up the execution of the search process.

[0055] As can be seen from the above several cases, the sorting operation, the selection operation, and other operations can be performed by the first processor 110 or the second processor 120.

[0056] The second processor 120 includes a vector unit 121 and a matrix unit 122, wherein the vector unit 121 can be configured to perform vector operations, and the matrix unit 122 can be configured to perform matrix operations.

[0057] The storage module 130 is configured to store the feature vectors of the base data, and store intermediate quantities generated in the retrieval process, for example, the distance matrix generated by the first processor 110, and can also be configured to store the distance table and the like.

[0058] The base data can be understood as a retrieval library, and can also be understood as the data set described above. The feature vector of the base data can be understood as a base feature obtained by feature extraction on the stored data, and each piece of stored data has a corresponding base feature.

[0059] Optionally, the stored data can be various types of data such as images, texts, voices, sequences and the like.

[0060] Optionally, the base feature described above can be obtained by using a feature extraction network to extract features from the base data.

[0061] Optionally, the feature extraction network described above can be a general neural network for extracting features, or a re-designed neural network capable of extracting features.

[0062] In some examples, the feature extraction network can include a residual network (resnet) and a fully connected layer (FC); in other examples, the feature extraction network can include a VGC16 and a fully connected layer.

[0063] Optionally, a neural network for calculating the distance between the base feature vector and the feature vector to be retrieved can be constructed, which can be referred to as a retrieval neural network for the sake of brevity. That is, the feature extraction network described above is a network for feature extraction, and the neural network here is a network for retrieving a base feature vector similar (close in distance) to the feature vector to be retrieved. The format of the weight matrix of the constructed neural network can be determined by the number of base feature vectors. For example, when the base feature vector length is 256 and there are a total of 300,000 base feature vectors, the weight matrix in the constructed neural network can be a three-dimensional matrix of 256*1*300,000.

[0064] Optionally, the retrieval neural network described above can be arranged in the second processor 120, and is configured to perform matrix / vector operations in the retrieval algorithm (retrieval process). In addition, the retrieval neural network described above can also be arranged in the storage module 130, and when the retrieval algorithm needs to be executed, the retrieval neural network can be read from the storage module 130.

[0065] Optionally, the feature extraction network can also be used to extract features of the data to be searched, and the features of the data to be searched are referred to as to-be-searched features. The feature extraction network can also be a general neural network for extracting features, or a re-designed neural network capable of extracting features.

[0066] In some examples, the feature extraction network can include a residual network and a fully connected layer; in other examples, the feature extraction network can include a VGC16 and a fully connected layer.

[0067] Generally, the feature extraction network used for extracting features of the base data is the same as the feature extraction network used for extracting features of the data to be searched. The same here refers to the same network structure and the same network parameters.

[0068] In some examples, the base features in the base data can be sorted according to the distances obtained by the search neural network (for example, the first distance below), and some base features can be selected as candidate results or search results. This process can be performed by the first processor 110 or the second processor 120. That is, the first processor 110 or the second processor 120 can sort the distances obtained by the search neural network in a certain order (for example, from large to small); select the top X distances from the sorted distances, and select the base features corresponding to the top X distances from the base data. The X base features can be used as search results, where X is a positive integer less than the total number of base features.

[0069] In other examples, the first processor 110 or the second processor 120 filters the distances obtained by the search neural network based on a preset threshold to eliminate distances exceeding the threshold (equivalent to eliminating base feature vectors with low similarity); then sorts the remaining distances in a certain order (for example, from large to small); select the top X distances from the sorted distances, and select the base features corresponding to the top X distances from the base data. The X base features are the search results, where X is a positive integer less than the total number of base features.

[0070] Optionally, in order to improve the retrieval efficiency, the base features in the base library can be clustered to obtain a plurality of cluster centers, and in the retrieval, the cluster centers with similar distances can be obtained first, and then the base features with similar distances can be further retrieved from the cluster centers with similar distances. This is equivalent to a hierarchical retrieval method, which can be understood as grouping the base features, that is, dividing the base features into groups, so that similar groups can be found first, and then similar base feature vectors can be found from the similar groups. Compared with the traditional one-to-one comparison with each base feature, the retrieval efficiency is higher. That is, the base library can include a plurality of cluster centers, and each cluster center includes a plurality of base features.

[0071] It should be noted that in the case where the base library includes a plurality of cluster centers, the second processor 120 can be used to calculate the distance between the to-be-retrieved feature and each cluster center, for example, the second distance (matrix / vector operation) in the following text, and then the distance is sorted (sorting operation) and some cluster centers are selected (selection operation), and then the first processor 110 is used to perform the query operation, at this time, the first processor 110 only needs to query the base features in the selected cluster centers. That is, in this process, matrix / vector operation, sorting operation and selection operation are also involved, the execution subject of these steps can be referred to the above, and will not be introduced again, the only difference is that the first processor 110 queries which base features when performing the query operation.

[0072] It can be understood that, Figure 1 The architecture of the retrieval system shown is only an example, and the retrieval system to which the retrieval method proposed in the present application can include more or fewer modules. For example, the neural network for feature extraction can be arranged in the second processor 120, or can be arranged in the storage module 130, or can be arranged in the Figure 1 In addition to the retrieval system shown, for example, in order to facilitate retrieval, the format of the base feature can be consistent with the format required by the retrieval neural network, but if the format of the base feature is inconsistent with the format of the retrieval neural network, the base feature needs to be converted from the current format (which can be referred to as the initial format) to the format that the retrieval neural network can read (which can be referred to as the target format), at this time, a format conversion module needs to be arranged, or the function of the format conversion module can be integrated on the first processor 110 or the second processor 120.

[0073] For example, when the first processor 110 generates the distance matrix, the distance matrix can be arranged in a format that can be supported by the neural network processor.

[0074] Figure 2 is a schematic structural diagram of the retrieval device of the embodiment of the present application. As shown in Figure 2As shown, the retrieval device 200 can include a main processor 210 and a neural network processor 230. The retrieval device 200 can also include a memory 220.

[0075] The main processor 210 can be a CPU. The main processor 210 can also be referred to as a host CPU.

[0076] The query operation can be implemented by the main processor 210. The matrix / vector operation can be implemented by the NPU 230. The sorting operation, the access to the distance table, the selection operation, and other operations can be implemented by either the main processor 210 or the neural network processor 230, that is, the host CPU can be the first processor 110, and the NPU can be the second processor 120. Figure 1 The NPU can be an example of the first processor 110. Figure 1 The second processor 120 can be an example of the second processor 120.

[0077] It should be noted that in the embodiments of the present application, the retrieval process includes multiple steps, which can be divided into a query operation, a matrix / vector operation, and other operations, which can include, for example, a sorting operation, a loading of a distance table, an access to the distance table, a selection operation, and the like. Among them, the query operation is relatively flexible and needs to be executed multiple times, and the NPU is not suitable for executing such operations, and the CPU is relatively suitable, so the query operation is allocated to the first processor (CPU) for execution. The matrix / vector operation is the operation that the NPU is best at, and the CPU cannot undertake complex matrix and vector operations relatively, so the matrix / vector operation is allocated to the second processor (NPU) for execution. As for other operations, either the first processor or the second processor can execute, for example, the idle processor in the two can be selected to execute, for example, the processor that undertakes fewer steps can be selected to execute, and for example, the same processor as the previous step can be selected to execute.

[0078] The neural network processor 230 is mainly used to complete the addition, subtraction, multiplication, and division numerical operations required for retrieval. For example, the NPU 230 completes the multiply-accumulate operation required for retrieval. The neural network processor 230 can be mounted as a coprocessor on the host CPU, and the host CPU inputs the data to be operated, such as the distance matrix described above.

[0079] The neural network processor 230 can include an input memory 201, a weight memory 202, an operation circuit 203, a controller 204, a storage unit controller 205, a unified memory 206, and an instruction fetch memory 207.

[0080] The unified memory 206, the input memory 201, the weight memory 202, and the instruction fetch memory 207 are on-chip memories.

[0081] The direct memory access controller (DMAC) 205 is mainly used to transfer input data and weight data in the memory 220 to the unified memory 206.

[0082] Further, the DMAC 205 can also be used to transfer weight data from the unified memory 206 to the weight memory 202, transfer input data from the unified memory 206 to the input memory 201, and transfer instructions to the instruction fetch memory 207.

[0083] The controller 204 executes instructions stored in the instruction fetch memory 207 to control the operation circuit to perform operations on the weights stored in the weight memory 202 and the input data in the input memory 201.

[0084] The NPU 230 controls the operation circuit 203 to extract matrix data in the weight memory 502 and the input memory 501 and perform matrix operations by executing corresponding instructions through the controller 204.

[0085] In some implementations, the operation circuit 203 can include a three-dimensional cube operation unit. The operation circuit 503 can also be a one-dimensional systolic array, a two-dimensional systolic array, or other electronic circuits capable of performing mathematical operations such as multiplication and addition.

[0086] For example, assuming there are an input matrix A, a weight matrix B, and an output matrix C. The operation circuit 203 takes corresponding data of the matrix B from the weight memory 202 and caches it on each operation unit of the operation circuit. The operation circuit 203 takes the matrix A data from the input memory 201 and performs matrix operations with the matrix B to obtain partial results or final results of the matrix, which are saved in the unified memory 206.

[0087] Figure 3 is a schematic flowchart of the retrieval method of an embodiment of the present application. The following describes Figure 3 each step of the method.

[0088] 301. Perform a table lookup operation in a retrieval process using a first processor.

[0089] Optionally, the first processor can be the first processor 110 shown in Figure 1 , or the main CPU 210 shown in Figure 2 .

[0090] It should be noted that the above retrieval process refers to retrieving corresponding base data from a base library using data to be retrieved, which can be data itself or a feature to be retrieved, and the base library stores base features corresponding to the base data in the base library.

[0091] Optionally, the table lookup operation can include: querying distance values between the to-be-retrieved feature and a plurality of base features in the base library, and generating a distance matrix of the to-be-retrieved feature and the plurality of base features, wherein the to-be-retrieved feature is used to represent a feature of the to-be-retrieved data, and the base feature is a feature of data in the base library.

[0092] Optionally, the distance values can be queried by querying a distance table.

[0093] Optionally, the distance matrix can be obtained by one-to-one comparison without clustering center division of the base library, which is equivalent to querying a distance table shared by all base features, reading corresponding distance values from the distance table, and generating the distance matrix by using the distance values.

[0094] Optionally, the base library can further include a plurality of clustering centers, each of the plurality of clustering centers including a plurality of base features. In this case, the second processor can be used to calculate second distances between the to-be-retrieved feature and the plurality of clustering centers; the first processor or the second processor can be used to select K clustering centers with the smallest second distances from the plurality of clustering centers, K being a positive integer; and the first processor can be used to query distance values between the to-be-retrieved feature and a plurality of base features in the K clustering centers in the form of a distance matrix.

[0095] That is, in the case where the base library further includes a plurality of clustering centers, distance values between the to-be-retrieved feature and each clustering center can be calculated first, and then distance values between base features in the selected clustering centers and the to-be-retrieved feature are queried, that is, the query operation can be performed only on the selected clustering centers, which can also be understood as equivalent to querying a distance table of the selected clustering centers and reading corresponding distance values from the distance table, and generating a distance matrix by using the distance values.

[0096] Optionally, a distance table of the base library can be obtained, and the distance table can be accessed when the query operation is performed. If clustering center division is performed, each clustering center has a corresponding distance table, and only the distance table of the selected clustering center needs to be accessed when the query operation is performed, that is, only distance values of base features in the selected clustering center can be accessed. If no clustering center division is performed and one-to-one comparison is used, the entire base library corresponds to a distance table, and the distance table needs to be accessed when the query operation is performed, that is, distance values of all base features can be accessed.

[0097] Optionally, the distance table can be calculated by the first processor or the second processor, that is, the operation of generating the distance table can be performed by the first processor or the second processor. It should be understood that the operation of obtaining the distance table and the operation of generating the distance matrix are different operation processes. After the feature to be searched is obtained, the distance table between the feature to be searched and the library features can be calculated, and the distance values between the feature to be searched and each library feature are included in the distance table. When the query operation is performed, the distance table can be queried to read out the distance values between a part of the library features and the feature to be searched to generate the distance matrix. It should also be understood that a distance table can be set for each library, which is equivalent to placing the distance values of all library features in the same table, or a distance table can be set for each cluster center, which is equivalent to placing the distance values of the library features of each cluster center in the table corresponding to each cluster center.

[0098] Optionally, the PQ cluster center can also be set, and when the distance table is generated, the distance between the feature to be searched and each PQ cluster center can be taken as the distance between the feature to be searched and the library features in each PQ cluster center, that is, the distance of each library feature in each PQ cluster center does not need to be calculated one by one, which is equivalent to sacrificing a certain accuracy to obtain higher operation efficiency. For example, assuming that a PQ cluster center in a certain library includes 10 library features, when the distance table is generated, the distance value between the feature to be searched and the PQ cluster center can be taken as the distance value between the feature to be searched and the 10 library features, respectively, which is equivalent to the 10 library features sharing the same distance value.

[0099] It should be noted that the PQ cluster center and the cluster center mentioned in the present application can be regarded as cluster centers generated by different clustering rules, wherein the PQ cluster center refers to a special cluster center that can be used for distance value sharing (coarse operation of distance value), and the other cluster center is only used for grouping all library features in the library. In some cases, the grouping requirement of the PQ cluster center can be relatively more stringent. Or it can be understood that the functions are different, the PQ cluster center can be regarded as being able to reduce the operation amount of the distance value, because each library feature does not need to be calculated, and the other cluster center can be regarded as being able to reduce the number of library features to be queried, because only the distance values of the part of the library features screened out need to be queried, and the two can respectively improve the search operation performance in different operation processes.

[0100] It should also be understood that the PQ cluster centers and other cluster centers can be set at the same time or only one of them can be set. Further, some specific values will be used for illustration to facilitate understanding, but it should be understood that this is not a limitation on the scheme of the present application. Assuming that a certain database includes 300 database features, when obtaining the distance table, in one way, the distances between the 300 database features and the feature to be searched can be directly calculated, which means that each distance value in the obtained distance table is calculated one by one. In another way, assuming that the 300 database features have been clustered into 50 PQ cluster centers, each of which includes 6 database features, and when obtaining the distance table, the distances between the 50 PQ cluster centers and the feature to be searched are calculated, which means that the 6 database features in each PQ cluster center share the same distance value in the obtained distance table. Both of the above cases can be regarded as generating only one distance table, so only one distance table needs to be queried when querying. In one implementation, the distance values of the 300 database features can be directly traversed when querying, which means accessing the above-mentioned one distance table. In another implementation, assuming that the 300 database features are clustered into 5 coarse cluster centers, each of which includes 60 database features, then the 5 coarse cluster centers correspond to a distance table, and each distance table can be obtained by using any of the above-mentioned ways. Assuming that 2 cluster centers are selected, only the distance tables of the 2 cluster centers need to be accessed when performing the query operation.

[0101] As can be seen from the above examples, the PQ cluster center can reduce the amount of calculation in the operation stage of obtaining the distance table, the coarse cluster center can reduce the access operation of the distance table in the access stage of the distance table in the query operation, and can reduce the amount of calculation when performing the matrix / vector operation on the subsequent distance matrix, both of which can improve the performance of the search algorithm.

[0102] 302, using the second processor to perform the matrix / vector operation in the search process.

[0103] Optionally, the second processor can be the second processor 120 shown in Figure 1 Optionally, the second processor can be the NPU 230 shown in Figure 2

[0104] Optionally, the above-mentioned matrix / vector operation can include: accumulating the distance matrix in step 301 to obtain a first distance between the feature to be searched and each of the plurality of database features.

[0105] Optionally, the accumulation of the distance matrix can be implemented by using vector operation, and the accumulation of the distance matrix can also be implemented by using matrix operation.

[0106] ​Optionally, when using matrix operation, the distance matrix can be multiplied by a weight matrix to obtain the distance, wherein the weight matrix can be set according to the library features. In this case, the operation efficiency can be improved by cooperation between the first processor and the second processor. For example, when the first processor generates the distance matrix, the distance matrix can be set to be consistent with the format required by the second processor, so that the second processor can directly operate the distance matrix, thereby saving processing steps. Since matrix operation is used, the vector unit is released, and other retrieval process operations can be performed using the vector unit, thereby achieving parallel algorithm and improving operation efficiency.

[0107] Optionally, the CPU needs to execute load and store instructions multiple times when performing table lookup operation. In order to reduce the instruction delay of write-after-read and reduce the number of execution of the above load and store instructions, the distance table can be preloaded, and the preloaded distance table can be resident in the L2 cache of the CPU. That is, when querying, the distance table needs to be accessed repeatedly multiple times, so that the distance table is more convenient for querying by preloading and resident. The distance table stores the distance values described above.

[0108] Optionally, the K cluster centers with the second minimum distance can be selected from the plurality of cluster centers, and K is a positive integer. This step can be performed by using the first processor (such as a CPU) or the second processor (such as the NPU 230). After the K cluster centers are selected, the first processor performs step 301. In the execution, in order to speed up the query, the distance values between the library features of the K cluster centers and the to-be-retrieved features can be queried. That is, the library features of other cluster centers are no longer queried, so that the amount of data queried can be effectively reduced, and the retrieval rate can be improved. However, it should be understood that the distance values of the library features of all cluster centers can also be queried. In this case, the query can be more thorough, and the retrieval accuracy can be higher. However, the amount of data queried is increased, so that the retrieval rate is reduced.

[0109] Optionally, the first processor or the second processor can be used to select the M library features with the minimum first distance as the retrieval result according to the first distance, and M is a positive integer.

[0110] In the retrieval method of the present application, the steps of the retrieval algorithm (retrieval process) are reasonably distributed to multiple processors for execution, so that the advantages of different types of processors can be brought into play, and the retrieval efficiency can be improved.

[0111] Figure 4 is a schematic flowchart of the retrieval method of the embodiment of the present application, and the following describes each step. Figure 4 ​

[0112] 401. Obtain a base feature.

[0113] The base feature is a feature of data in the base.

[0114] Optionally, the base can include a plurality of cluster centers, each of which can include a plurality of base features.

[0115] Optionally, the step 401 can be performed by the first processor 110 or the second processor 120 in the processor 100, or the main CPU 210 in the processor 200, or the NPU 230 in the processor 300. Figure 1 Figure 2 Figure 2

[0116] The data in the base can be referred to as base data. In general, the base data is collected in advance or collected.

[0117] The base data can include at least one of the following data types: image, text, voice, sequence, etc.

[0118] Optionally, the base data can be feature extracted to obtain the base feature, and the base feature can be clustered to obtain a plurality of cluster centers. That is, the base can include a plurality of cluster centers, each of which can include a plurality of base features.

[0119] Optionally, the cluster centers can be further subdivided, that is, a plurality of hierarchical cluster centers can be set. For example, the base can be set to include a plurality of first-level cluster centers, each first-level cluster center includes a plurality of second-level cluster centers, and each second-level cluster center includes a plurality of base features. When searching, the first-level cluster center closest to the distance can be searched first, then the second-level cluster center closest to the distance is found from the first-level cluster center closest to the distance, and then the base feature closest to the distance is found from the second-level cluster center closest to the distance. Different distance thresholds can be set to filter out the corresponding cluster centers or base features.

[0120] ​​​Optionally, the base library features or the cluster centers can be quantization coded. For example, the distance described above can use 1 byte to represent the coding of 256 cluster centers, so that each 128-dimensional floating point vector can be represented by only 4 bytes. That is, the cluster centers are numbered, and each base library feature in each cluster center is numbered, and the number of the base library feature can indicate which cluster center the base library feature belongs to.

[0121] The data in the base library can include images, and when the data is images, the images can include one or more of a human face image, a fingerprint image, a license plate image, a vehicle image, a human image, and the like. The data in the base library can be updated as needed, such as adding, deleting, or replacing data or base library feature vectors corresponding to the data.

[0122] In some implementations, each data in the base library can be input into a neural network for feature extraction, and the extracted features are referred to as base library features. The feature extraction network can use a general neural network for feature extraction, or a redesigned neural network capable of extracting features. For example, when the image library contains 300,000 base library images, the neural network for feature extraction can include Resnet50 or VGG16. In other implementations, the base library features can be received or copied directly from other devices or systems.

[0123] One form of base library features is a matrix, and if the matrix corresponding to the base library features is a one-dimensional matrix, the form of the base library features is a vector, which can be referred to as a base library feature vector. In one example, a base library feature vector can be 256-dimensional, with each dimension being a 32-bit floating point number (fp32) data.

[0124] As described above, the base library data can be of different data types, and when the base library data is an image, it can be referred to as a base library image. The base library image can include a human face image, a fingerprint image, a license plate image, a vehicle image, or a human image, and the like. The base library features can be regarded as features obtained after feature extraction of the base library image.

[0125] The base library images can be numbered, that is, each base library image is assigned an index number for identifying each base library image. After obtaining the base library features of the base library images, each base library feature can also be assigned an index number, and the index number of the base library feature can be the same as the index number of the corresponding base library image.

[0126] Optionally, to facilitate retrieval, the format of the base library features can be consistent with the format of the data features to be retrieved.

[0127] Optionally, when the format of the data feature to be retrieved is inconsistent with the format of the base library feature, a format conversion method can be used to make them consistent.

[0128] Optionally, in order to facilitate retrieval, the format of the base library feature can be made consistent with the format required by the retrieval neural network. If the format of the obtained base library feature is inconsistent with the format of the retrieval neural network, the base library feature needs to be converted from the current format (which can be referred to as the initial format) to a format (which can be referred to as the target format) that can be read by the retrieval neural network.

[0129] 402. Obtain a feature to be retrieved, which is a feature of data to be retrieved.

[0130] Optionally, the feature to be retrieved can be obtained directly, or the feature to be retrieved can be obtained by feature extraction on the data to be retrieved. For example, the feature to be retrieved is obtained by feature extraction on the data to be retrieved by the feature extraction network described above. The feature of the data to be retrieved can be referred to as the feature to be retrieved. One form of the feature to be retrieved is a vector, which can be referred to as a feature vector to be retrieved.

[0131] Optionally, the first processor 110 or the second processor 120 in Figure 1 or the main CPU 210 in Figure 2 or the NPU 230 in Figure 2 can be used to perform step 402. Step 402 can also be performed by an acquisition unit independent of the chip. That is, there is no limitation on the execution subject of step 402. The execution subjects of steps 401 and 402 can be the same or different. In addition, the execution of steps 401 and 402 is not limited in order, and can be performed simultaneously or not simultaneously, and the order of execution is not limited. It should also be understood that the base library feature and the feature to be retrieved can have been obtained. Therefore, steps 401 and 402 can not be executed, and the processor directly performs operations such as query, calculation, and sorting on the prepared data. That is, steps 401 and 402 are not necessary steps of the technical solution of the present application.

[0132] In one example, a feature vector to be retrieved can be 256-dimensional, and each dimension is a 32-bit floating-point number (fp32) data.

[0133] The data to be retrieved can be any type of data described above. When the data to be retrieved is an image, it can be referred to as a retrieval image. The retrieval image can include a face image, a fingerprint image, a license plate image, a vehicle image, or a human image, etc. The retrieval image can be one or a batch, i.e., multiple images.

[0134] The images to be searched can be numbered, that is, each image to be searched is assigned an index number for identifying each image to be searched. After obtaining the features to be searched of the image to be searched, each feature to be searched can also be assigned an index number, which can be the same as the index number of the corresponding image to be searched.

[0135] 403. Query the distance values between the features to be searched and the features in the base library to generate a distance matrix.

[0136] Optionally, the first processor 110 in Figure 1 or the main CPU 210 in Figure 2 can be used to perform step 403.

[0137] Optionally, the distance matrix can be generated by querying the distance values between the features to be searched and each feature in the base library.

[0138] Optionally, the distance matrix can be generated by querying the distance table. It should be noted that the above is a one-to-one comparison method to obtain the distance matrix without clustering center division, which is equivalent to querying the distance table including all distance values of the features in the base library, reading the corresponding distance values from the distance table, and generating the distance matrix using the read distance values. However, it should be understood that, as described above, the base library can also include a plurality of clustering centers, and each clustering center can include a plurality of features in the base library. In this case, the distance values between the features to be searched and each clustering center can be calculated first, and then the distance values between the features to be searched and the features in the selected clustering centers can be queried, which is equivalent to querying the distance table of the features in the selected clustering centers and reading the corresponding distance values from the distance table, and generating the distance matrix using the read distance values.

[0139] Optionally, the distance table of the base library can be obtained, and then the distance table can be accessed when performing the query operation. If the clustering center division is performed, each clustering center has a corresponding distance table, and only the distance table of the selected clustering center needs to be accessed when performing the query operation, that is, only the distance values of the features in the selected clustering center can be accessed. If the clustering center division is not performed and the one-to-one comparison method is used, the entire base library corresponds to a distance table, and the distance table needs to be accessed when performing the query operation, that is, all distance values of the features in the base library can be accessed.

[0140] Optionally, the second distance between the feature to be searched and the cluster center can be calculated first, and this step can be performed by the first processor (e.g., CPU) or the second processor (e.g., NPU 230), but since this calculation is a vector operation, it can be performed by the second processor to improve the operation speed.

[0141] Optionally, the K cluster centers with the smallest second distance can be selected from the plurality of cluster centers, K being a positive integer, and this step can be performed by the first processor (e.g., CPU) or the second processor (e.g., NPU 230). After the K cluster centers are selected, the first processor performs step 403, and in the execution, in order to speed up the query, only the distance values between the library features in the K cluster centers and the feature to be searched can be queried. This is equivalent to no longer querying the library features of other cluster centers, which can effectively reduce the amount of data queried and improve the search speed. However, it should be understood that the distance values of the library features of all cluster centers can also be queried, which is equivalent to a more thorough query with higher search accuracy, but the search speed will be reduced due to the increased amount of data queried.

[0142] Optionally, in order to facilitate the operation of the second processor, the distance matrix generated in step 403 can be set to be consistent with the format required by the second processor. That is, in step 403, the distance matrix is set to the same format as in step 404 to facilitate the execution of step 404.

[0143] Optionally, when performing a table lookup operation, the CPU needs to execute a load instruction and a save instruction multiple times. In order to reduce the instruction delay of write-after-read and reduce the number of times the above-mentioned load and save instructions are executed, the distance table can be preloaded, and the preloaded distance table can be resident in the CPU's level 2 cache. That is, when querying, the distance table needs to be accessed repeatedly multiple times, so by preloading and resident, the distance table is more convenient to query. The distance table stores the distance values described above.

[0144] 404, Accumulate the distance matrix to obtain the first distance between the feature to be searched and the library feature.

[0145] Optionally, the second processor 120 in Figure 1 or the NPU 230 in Figure 2 can be used to perform step 404.

[0146] Optionally, step 404 can be performed by the vector unit 121 in the second processor 120, in which case the second processor 120 is equivalent to executing a vector accumulation instruction to complete the accumulation of the distance matrix. After vector accumulation, a sorting instruction can also be executed by the vector unit 121 to obtain the M library features with the smallest first distance as the retrieval result, M being a positive integer.

[0147] Optionally, step 404 can also be performed by the matrix unit 122 in the second processor 120, in which case the second processor 120 is equivalent to using matrix operation to implement the accumulation of the distance matrix.

[0148] Optionally, the distance matrix and the weight matrix can be multiplied to complete the accumulation by matrix multiplication. Matrix multiplication has a faster operation speed than the vector accumulation method, and the weight matrix can be used multiple times after being set, so it does not need to be loaded frequently.

[0149] It should be noted that the matrix operation method has higher requirements for the data arrangement of the distance matrix than the vector operation method, but the format conversion can be completed at the same time as the CPU generates the distance matrix during the query operation, without additional overhead. That is, on the premise that the first processor is used to set the distance matrix to be consistent with the format of the second processor as described above, the second processor 120 does not need additional processing when using the matrix unit 122 to perform matrix operation to accumulate the distance matrix. In the case of using the matrix unit to accumulate, the sorting operation can be performed simultaneously by using the vector unit, thereby improving the parallelism of the algorithm and further improving the performance.

[0150] 405. According to the first distance, the M library features with the smallest first distance are selected as the retrieval result, M being a positive integer.

[0151] Optionally, step 405 can be performed by the first processor 110 or the second processor 120 in Figure 1 or the main CPU 210 in Figure 2 or the NPU 230 in Figure 2 .

[0152] In the retrieval method of the present application, the steps of the retrieval algorithm are reasonably distributed to various chips for execution, so that the advantages of different types of chips can be brought into play to improve the retrieval efficiency.

[0153] Figure 5 is a schematic structural diagram of a retrieval device of an embodiment of the present application. The retrieval device 500 comprises a first processor 510 and a second processor 520.

[0154] Optionally, the device 500 can be a chip, which can be called a retrieval chip. The retrieval chip may include a first chip and a second chip, wherein the first chip corresponds to the first processor 510 and is a CPU chip, and the second chip corresponds to the second processor and is an NPU chip.

[0155] The device 500 can implement the retrieval method described above. For example, the first processor 510 can be used to execute... Figure 3 As shown in step 301, the second processor 520 can be used to execute... Figure 3 Step 302 is shown. For example, the first processor 510 can be used to execute... Figure 4 As shown in step 403, the second processor 520 can be used to execute... Figure 4 As shown in step 404, both the first processor 510 and the second processor 520 can be used to execute... Figure 4 Steps 401, 402, and 405 are shown.

[0156] Optionally, the first processor 510 may correspond to Figure 1 The first processor 110 shown can also correspond to Figure 2 The main CPU shown is 210.

[0157] Optionally, the second processor 520 may correspond to Figure 1 The second processor 120 shown can also correspond to Figure 2 The NPU 230 is shown.

[0158] Optionally, the retrieval device 500 may also include a storage module 530, which may be integrated into the first processor 510, integrated into the second processor 520, or be a device in the retrieval device 500 that is independent of the first processor 510 and the second processor 520, or be a device that is independent of the retrieval device 500.

[0159] Optionally, the storage module 530 can correspond to Figure 1 The storage module 130 shown can also correspond to Figure 2 The memory 220 is shown.

[0160] Figure 6 This is a schematic structural diagram of the retrieval device according to an embodiment of this application. Figure 6 As shown, device 3000 includes memory 3001, processor 3002, and communication interface 3003. One example of device 3000 is a chip. Another example of device 3000 is a computing device.

[0161] The memory 3001, the processor 3002 and the communication interface 3003 can communicate through the bus 3004. The memory 3001 stores executable code, and the processor 3002 reads the executable code in the memory 3001 to execute the corresponding method. The memory 3001 can also include software modules required by other running processes such as operating systems. The operating system can be LINUX TM , UNIX TM , WINDOWS TM , etc.

[0162] For example, the executable code in the memory 3001 is used to implement the method shown in 3, and the processor 3002 reads the executable code in the memory 3001 to execute the method of retrieval described above.

[0163] The processor 3002 can include a CPU and an NPU, or the exemplary structure of the processor 3002 is as shown in Figure 1 or Figure 2 The memory 3001 can include a volatile memory such as a random access memory (RAM). The memory 3001 can also include a non-volatile memory such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid state disk (SSD).

[0164] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0166] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0167] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0168] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0169] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and various program code storage media.

[0170] The above is merely specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A retrieval device, characterized in that, include: The first processor is used to perform the table lookup operation during the retrieval process. The first processor is a general-purpose central processing unit (CPU). The second processor is used to perform matrix / vector operations in the retrieval process. The second processor is a neural network processor (NPU). Specifically, the table lookup operation includes: querying the distance values ​​between the feature to be retrieved and multiple base database features in the base database, generating a distance matrix between the feature to be retrieved and the multiple base database features, wherein the feature to be retrieved is used to represent the features of the data to be retrieved, and the base database features are the features of the data in the base database; The matrix / vector operation specifically includes matrix operation and vector operation: the matrix operation includes summing the distance matrix to obtain a first distance between the feature to be retrieved and each of the multiple base database features; the vector operation includes a sorting operation. The first processor is further configured to adjust the data arrangement of the distance matrix while performing the table lookup operation, so that the format of the distance matrix generated by the first processor meets the requirements of the second processor. The second processor includes a matrix unit and a vector unit. The matrix unit is used to perform the matrix operation, and the vector unit is used to perform the vector operation. The matrix operation performed by the matrix unit and the vector operation performed by the vector unit are executed in parallel.

2. The retrieval device as described in claim 1, characterized in that, The first processor or the second processor is used for: Based on the first distance, the M base database features with the smallest first distance are selected as the retrieval results, where M is a positive integer.

3. The retrieval device as described in claim 1 or 2, characterized in that, The base database also includes multiple cluster centers, each of which includes multiple base database features. The second processor core is further configured to calculate a second distance between the feature to be retrieved and the multiple cluster centers. The first processor or the second processor is further configured to select the K cluster centers with the smallest second distance from the plurality of cluster centers, where K is a positive integer; The first processor is specifically used to query the distance values ​​between the feature to be retrieved and multiple base features in the K cluster centers to generate the distance matrix.

4. The retrieval device as described in claim 1 or 2, characterized in that, The step of summing the distance matrices to obtain the first distance between the feature to be retrieved and each of the multiple base database features specifically includes: The first distance is obtained by multiplying the distance matrix by the weight matrix, wherein the weight matrix is ​​set according to the base database features.

5. A retrieval method, characterized in that, include: The table lookup operation in the retrieval process is performed using a first processor, which is a general-purpose central processing unit (CPU). The matrix / vector operations in the retrieval process are performed using a second processor, which is a neural network processor (NPU). Specifically, the table lookup operation includes: querying the distance values ​​between the feature to be retrieved and multiple base database features in the base database, generating a distance matrix between the feature to be retrieved and the multiple base database features, wherein the feature to be retrieved is used to represent the features of the data to be retrieved, and the base database features are the features of the data in the base database; The matrix / vector operation specifically includes matrix operation and vector operation: the matrix operation includes summing the distance matrix to obtain a first distance between the feature to be retrieved and each of the multiple base database features; the vector operation includes a sorting operation. The first processor is further configured to adjust the data arrangement of the distance matrix while performing the table lookup operation, so that the format of the distance matrix generated by the first processor meets the requirements of the second processor. The second processor includes a matrix unit and a vector unit. The matrix unit is used to perform the matrix operation, and the vector unit is used to perform the vector operation. The matrix operation performed by the matrix unit and the vector operation performed by the vector unit are executed in parallel.

6. The retrieval method as described in claim 5, characterized in that, The method further includes: The first processor or the second processor selects the M base database features with the smallest first distance as the retrieval results based on the first distance, where M is a positive integer.

7. The retrieval method as described in claim 5 or 6, characterized in that, The base database also includes multiple cluster centers, each of which includes multiple features of the base database. The retrieval method further includes: The second processor calculates a second distance between the feature to be retrieved and the plurality of cluster centers; The first processor or the second processor selects the K cluster centers with the smallest second distance from the plurality of cluster centers, where K is a positive integer; The first processor queries the distance values ​​between the feature to be retrieved and multiple base features in the K cluster centers to generate the distance matrix.

8. The retrieval method as described in claim 5 or 6, characterized in that, The step of summing the distance matrices to obtain a first distance between the feature to be retrieved and each of the multiple base database features includes: The first distance is obtained by multiplying the distance matrix by the weight matrix, wherein the weight matrix is ​​set according to the base database features.

9. A retrieval device, characterized in that, include: Memory, used to store instructions; A processor coupled to the memory is configured to execute instructions stored in the memory to cause the apparatus to perform the retrieval method as described in any one of claims 5 to 8.

10. A computer-readable medium, characterized in that, Includes instructions that, when executed on a processor, cause the processor to perform the retrieval method as described in any one of claims 5 to 8.

Images