Image retrieval method, system and device

Abstract

The embodiment of the invention discloses an image retrieval method, and the method comprises the steps: carrying out the matrix multiplication calculation of a left matrix constructed based on a query vector and a right matrix constructed based on a plurality of reference vectors, so as to obtain a Hamming distance between the query vector and each reference vector. According to the scheme provided by the embodiment of the invention, the Hamming distance between the vectors can be obtained by executing the matrix multiplication, so that the calculation amount required for calculating the Hamming distance and the duration required for calculating the Hamming distance can be reduced, and the image retrieval efficiency is improved. In addition, the embodiment of the invention further discloses an image retrieval system and device and a readable storage medium, and the technical effects of the image retrieval system and device and the readable storage medium correspond to the technical effects of the method.

Description

[0001] This application is a divisional application. The application number of the original application is 202310090663.6, and the original application date is January 29, 2023. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computers, and in particular to an image retrieval method, system and device. Background Art

[0003] The Hamming distance between two vectors of the same dimension is the number of different bits between the two vectors. When calculating the Hamming distance between two vectors through a program, each bit of the two vectors is XORed and the results of the XOR calculation are added to obtain the Hamming distance between the two vectors. In the case where the number of bits of the vector is very large, multiple loop calculations are required during the above calculation process, and a large amount of data is transmitted between the memory and the central processing unit (CPU), resulting in a huge amount of calculation and a long calculation time.

[0004] In the process of image retrieval, the Hamming distance can be used to measure the similarity between the vector corresponding to the image to be retrieved and the vector corresponding to the image in the gallery. For example, when a user inputs an image to be retrieved, the system obtains the Hamming distance between the vector corresponding to the image in the gallery and the vector corresponding to the image to be retrieved, and outputs at least one image in the gallery or the serial number corresponding to at least one image according to the order of the Hamming distance.

[0005] How to reduce the time required to calculate the Hamming distance between vectors and improve the efficiency of image retrieval is an issue worthy of attention. Summary of the invention

[0006] The present application provides a method, system and device for image retrieval. The Hamming distance between vectors can be obtained by performing matrix multiplication, thereby reducing the amount of calculation required to calculate the Hamming distance and the time required to calculate the Hamming distance, thereby improving the efficiency of image retrieval.

[0007] The first aspect of the embodiment of the present application provides a method for image retrieval, including: obtaining a query vector and a plurality of reference vectors, wherein the query vector is obtained based on the image to be retrieved, and the reference vector is obtained based on the image included in the gallery. Specifically, the query vector can be a vector output by the feature extraction model after the image to be retrieved is input into the feature extraction model. The reference vector can be a vector output by the feature extraction model after the image included in the gallery is input into the feature extraction model. The embodiment of the present application does not limit the specific type of the feature extraction model, and any model that can convert an image into a vector representation can be adopted by the embodiment of the present application. Matrix multiplication is performed on a left matrix constructed based on the query vector and a right matrix constructed based on a plurality of reference vectors to obtain the Hamming distance between the query vector and each reference vector. Assume that the query vector and the reference vector are both N-dimensional vectors, and N is a positive integer. There are a total of M query vectors and P reference vectors, then the vector calculation unit can construct a matrix of M rows and N columns according to the query vector, and use it as the left matrix, and construct a matrix of P rows and N columns according to the reference vector, and transpose the matrix of P rows and N columns, and obtain a matrix of N rows and P columns as the right matrix. The solution provided by the embodiment of the present application converts the process of calculating the Hamming distance into the process of calculating the matrix multiplication, so the process of calculating the Hamming distance can be completed by the matrix calculation unit. The search results are obtained according to the sorting results of multiple Hamming distances, and the search results include images in the gallery similar to the image to be retrieved. The multiple Hamming distances can be sorted in order from small to large or from large to small, and the serial numbers of the K reference vectors with the smallest Hamming distance between the query vector are output, or the images in the gallery corresponding to the K reference vectors with the smallest Hamming distance between the query vector are output, and K is a positive integer. The solution provided in the first aspect converts the process of calculating the Hamming distance into a matrix multiplication operation, so the process of calculating the Hamming distance can be completed by the matrix multiplication calculation unit. The process of calculating the Hamming distance is completed by the matrix multiplication calculation unit, and there is no need to perform cumbersome processes, such as there is no need to perform XOR calculations on each bit of the two vectors in turn, and add the results of the XOR calculations to obtain the Hamming distance between the two vectors. The matrix multiplication calculation unit can be specifically used to perform matrix multiplication calculations. Compared with using CPU resources to calculate the Hamming distance, calculating the Hamming distance by the matrix multiplication calculation unit can improve the calculation efficiency. This can reduce the amount of calculation required to calculate the Hamming distance between the query vector and the reference vector during the image retrieval process, reduce the time required to calculate the Hamming distance between the query vector and the reference vector, and improve the efficiency of image retrieval.

[0008] In a possible implementation of the first aspect, the query vector and the reference vector are both N-dimensional vectors, where N is a positive integer, and matrix multiplication is performed on a left matrix constructed based on the query vector and a right matrix constructed based on multiple reference vectors to obtain the Hamming distance between the query vector and each reference vector, including: performing matrix multiplication on the left matrix and the right matrix to obtain a first calculation result. The Hamming distance between the query vector and each reference vector is obtained based on the first calculation result and N. In this implementation, a specific method for obtaining the Hamming distance based on matrix multiplication is provided, which increases the diversity of the scheme.

[0009] In a possible implementation of the first aspect, the query vector and the reference vector are both binary vectors, each dimension of the binary vector has a value of 0 or 1, each element in the left matrix has a value of -a or a, a is a positive integer, the position of the element with a value of -a in the left matrix is ​​obtained based on the position of the element with a value of 0 in the query vector, the position of the element with a value in the left matrix is ​​obtained based on the position of the element with a value of 1 in the query vector, and each element in the right matrix has a value of -a or a, wherein the position of the element with a value of -a in the right matrix is ​​obtained based on the position of the element with a value of 0 in each reference vector, and the position of the element with a value in the right matrix is ​​obtained based on the position of the element with a value of 1 in each reference vector. In this implementation, a specific way of constructing the left matrix and the right matrix is ​​given, which increases the diversity of the scheme.

[0010] In a possible implementation of the first aspect, the value of a is 1.

[0011] In a possible implementation of the first aspect, the retrieval results are obtained based on the sorting results of multiple Hamming distances, including: copying the serial numbers corresponding to the multiple Hamming distances to a target matrix of N+1 rows and K columns, K is a positive integer, the values ​​of the Hamming distances corresponding to the serial numbers stored in each row of the target matrix are the same, and the value of the Hamming distance corresponding to any serial number stored in the previous row of N+1 rows is less than the value of the Hamming distance corresponding to any serial number stored in the next row. K serial numbers are obtained based on the target matrix, and the K serial numbers are used to obtain the retrieval results. In this implementation, the sorting results of multiple Hamming distances can be completed by a vector calculation unit, which can improve the efficiency of sorting multiple Hamming distances and further improve the efficiency of image retrieval.

[0012] In a possible implementation of the first aspect, the serial numbers corresponding to the multiple Hamming distances are copied to the target matrix of N+1 rows and K columns, including: traversing the multiple Hamming distances to count the number of current target Hamming distances, where the target Hamming distance is any one of the multiple Hamming distances. The offset vector of the target Hamming distance is obtained according to the number of current target Hamming distances, the value of the target Hamming distance, and K. The serial number corresponding to the target Hamming distance is copied to a specified position of the target matrix, where the specified position is obtained based on the offset vector. In this implementation, a specific way of sorting the Hamming distances is given, which increases the diversity of the scheme.

[0013] In a possible implementation of the first aspect, copying the serial number corresponding to the target Hamming distance to the specified position of the target matrix includes: calling a scatter instruction to copy the serial number corresponding to the target Hamming distance to the specified position of the target matrix.

[0014] In a possible implementation of the first aspect, the method further includes dividing the multiple reference vectors into multiple groups, each group includes at least one reference vector, and the multiple groups include a first group and a second group. Copying the serial numbers corresponding to the multiple Hamming distances to the target matrix of N+1 rows and K columns includes: when obtaining the Hamming distances between the query vector and each reference vector in the second group, copying the serial numbers corresponding to the Hamming distances between the query vector and each reference vector in the first group to the target matrix of N+1 rows and K columns. In this implementation, the matrix calculation unit and the vector calculation unit can perform an asynchronous pipeline execution scheme to improve the performance of image retrieval.

[0015] The second aspect of the embodiment of the present application provides an image retrieval device, which includes a vector calculation unit and a matrix calculation unit. The vector calculation unit is used to obtain a query vector and multiple reference vectors, the query vector is obtained based on the image to be retrieved, and the reference vector is obtained based on the images included in the gallery. The matrix calculation unit is used to perform matrix multiplication calculations on a left matrix constructed by the vector calculation unit based on the query vector and a right matrix constructed by the vector calculation unit based on the multiple reference vectors to obtain the Hamming distance between the query vector and each reference vector. The vector calculation unit is also used to obtain a retrieval result based on the sorting results of the multiple Hamming distances, and the retrieval result includes images in the gallery that are similar to the image to be retrieved.

[0016] In a possible implementation of the second aspect, the query vector and the reference vector are both N-dimensional vectors, where N is a positive integer, and the matrix calculation unit is specifically used to: perform matrix multiplication calculation on the left matrix and the right matrix to obtain a first calculation result. According to the first calculation result and N, the Hamming distance between the query vector and each reference vector is obtained.

[0017] In a possible implementation of the second aspect, the query vector and the reference vector are both binary vectors, each dimension of the binary vector has a value of 0 or 1, each element in the left matrix has a value of -a or a, a is a positive integer, the position of the element with a value of -a in the left matrix is ​​obtained based on the position of the element with a value of 0 in the query vector, the position of the element with a value of a in the left matrix is ​​obtained based on the position of the element with a value of 1 in the query vector, and the value of each element in the right matrix is ​​-a or a, wherein the position of the element with a value of -a in the right matrix is ​​obtained based on the position of the element with a value of 0 in each reference vector, and the position of the element with a value in the right matrix is ​​obtained based on the position of the element with a value of 1 in each reference vector.

[0018] In a possible implementation of the second aspect, the value of a is 1.

[0019] In a possible implementation of the second aspect, the vector calculation unit is specifically used to: copy the serial numbers corresponding to the multiple Hamming distances to a target matrix of N+1 rows and K columns, K is a positive integer, the values ​​of the Hamming distances corresponding to the serial numbers stored in each row of the target matrix are the same, and the value of the Hamming distance corresponding to any serial number stored in the previous row of the N+1 rows is less than the value of the Hamming distance corresponding to any serial number stored in the next row. K serial numbers are obtained according to the target matrix, and the K serial numbers are used to obtain the search results.

[0020] In a possible implementation of the second aspect, the vector calculation unit is specifically configured to: traverse multiple Hamming distances to count the number of current target Hamming distances, where the target Hamming distance is any one of the multiple Hamming distances.

[0021] The offset vector of the target Hamming distance is obtained according to the number of the current target Hamming distance, the value of the target Hamming distance, and K. The serial number corresponding to the target Hamming distance is copied to the specified position of the target matrix, and the specified position is obtained based on the offset vector.

[0022] In a possible implementation of the second aspect, the vector calculation unit is specifically used to: call a scatter instruction to copy the sequence number corresponding to the target Hamming distance to a specified position of the target matrix.

[0023] In a possible implementation of the second aspect, the device further includes a main control logic unit, the main control logic unit is used to divide the multiple reference vectors into multiple groups, each group includes at least one reference vector, and the multiple groups include a first group and a second group. The vector calculation unit is specifically used to copy the sequence number corresponding to the Hamming distance between the query vector and each reference vector in the first group to a target matrix with N+1 rows and K columns when the matrix calculation unit obtains the Hamming distance between the query vector and each reference vector in the second group.

[0024] In a possible implementation of the second aspect, the utilization rate of the matrix calculation unit exceeds 60%. In the solution provided in the embodiment of the present application, the Hamming distance is calculated by the matrix calculation unit during the image retrieval process, so the utilization rate of the matrix calculation unit is relatively high during the image retrieval process, and the utilization rate of the matrix calculation unit can exceed 60% without flow control.

[0025] A third aspect of an embodiment of the present application provides an image retrieval system, the system comprising a vector calculation unit and a matrix calculation unit, the vector calculation unit being used to obtain a query vector and multiple reference vectors, the query vector being obtained based on an image to be retrieved, and the reference vector being obtained based on an image included in a gallery. The matrix calculation unit is used to perform matrix multiplication calculations on a left matrix constructed by the vector calculation unit based on the query vector and a right matrix constructed by the vector calculation unit based on multiple reference vectors, so as to obtain the Hamming distance between the query vector and each reference vector. The vector calculation unit is also used to obtain retrieval results based on the sorting results of multiple Hamming distances, the retrieval results including images in the gallery that are similar to the image to be retrieved.

[0026] In a possible implementation of the third aspect, the query vector and the reference vector are both N-dimensional vectors, where N is a positive integer, and the matrix calculation unit is specifically used to: perform matrix multiplication calculation on the left matrix and the right matrix to obtain a first calculation result. According to the first calculation result and N, the Hamming distance between the query vector and each reference vector is obtained.

[0027] In a possible implementation of the third aspect, the query vector and the reference vector are both binary vectors, each dimension of the binary vector has a value of 0 or 1, each element in the left matrix has a value of -a or a, a is a positive integer, the position of the element with a value of -a in the left matrix is ​​obtained based on the position of the element with a value of 0 in the query vector, the position of the element with a value of a in the left matrix is ​​obtained based on the position of the element with a value of 1 in the query vector, and each element in the right matrix has a value of -a or a, wherein the position of the element with a value of -a in the right matrix is ​​obtained based on the position of the element with a value of 0 in each reference vector, and the position of the element with a value in the right matrix is ​​obtained based on the position of the element with a value of 1 in each reference vector.

[0028] In a possible implementation of the third aspect, the value of a is 1.

[0029] In a possible implementation of the third aspect, the vector calculation unit is specifically used to: copy the serial numbers corresponding to the multiple Hamming distances to a target matrix of N+1 rows and K columns, K is a positive integer, the values ​​of the Hamming distances corresponding to the serial numbers stored in each row of the target matrix are the same, and the value of the Hamming distance corresponding to any serial number stored in the previous row of the N+1 rows is less than the value of the Hamming distance corresponding to any serial number stored in the next row. K serial numbers are obtained according to the target matrix, and the K serial numbers are used to obtain the search results.

[0030] In a possible implementation of the third aspect, the vector calculation unit is specifically used to: traverse multiple Hamming distances to count the number of current target Hamming distances, where the target Hamming distance is any one of the multiple Hamming distances.

[0031] The offset vector of the target Hamming distance is obtained according to the number of the current target Hamming distance, the value of the target Hamming distance, and K. The serial number corresponding to the target Hamming distance is copied to the specified position of the target matrix, and the specified position is obtained based on the offset vector.

[0032] In a possible implementation of the third aspect, the vector calculation unit is specifically used to: call a scatter instruction to copy the sequence number corresponding to the target Hamming distance to a specified position of the target matrix.

[0033] In a possible implementation of the third aspect, the system further includes a main control logic unit, the main control logic unit is used to divide the multiple reference vectors into multiple groups, each group includes at least one reference vector, and the multiple groups include a first group and a second group. The vector calculation unit is specifically used to copy the sequence number corresponding to the Hamming distance between the query vector and each reference vector in the first group to a target matrix with N+1 rows and K columns when the matrix calculation unit obtains the Hamming distance between the query vector and each reference vector in the second group.

[0034] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, including a program, which, when executed on a computer, enables the computer to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0035] A fifth aspect of an embodiment of the present application provides a computer program product, which includes computer software instructions. The computer software instructions can be loaded by a processor to implement the method of the above-mentioned first aspect or any one of the implementation methods of the first aspect.

[0036] For the specific implementation steps of the second to fifth aspects of the present application and various possible implementation methods, as well as the beneficial effects brought about by each possible implementation method, reference can be made to the description of the first aspect and the various possible implementation methods in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flowchart of the process of image retrieval using Hamming distance;

[0038] Figure 2 A schematic diagram of the architecture of an image retrieval system provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of a flow chart of an image retrieval method provided in an embodiment of the present application;

[0040] Figure 4 Another schematic diagram of a flow chart of an image retrieval method provided in an embodiment of the present application;

[0041] Figure 5 Another schematic diagram of a flow chart of an image retrieval method provided in an embodiment of the present application;

[0042] Figure 6 Another schematic diagram of a flow chart of an image retrieval method provided in an embodiment of the present application;

[0043] Figure 7 Another schematic diagram of a flow chart of an image retrieval method provided in an embodiment of the present application;

[0044] Figure 8 A schematic diagram showing the advantages of the image retrieval method provided in the embodiment of the present application;

[0045] Fig. 9 A timing flow chart of the image retrieval method provided in an embodiment of the present application;

[0046] Fig.10 Another timing flow chart of the image retrieval method provided in the embodiment of the present application;

[0047] Fig.11 Another timing flow chart of the image retrieval method provided in the embodiment of the present application;

[0048] Fig.12 A schematic diagram of the structure of an image retrieval device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] Data retrieval refers to the process of searching the data features stored in the database with the query data as the retrieval object to obtain the results that match the query data. Data retrieval can be divided into image retrieval (image search), video retrieval, and text retrieval according to the different retrieval objects. Furthermore, image retrieval can be further subdivided into face recognition, image infringement judgment, product recommendation, etc. according to the application scenario, among which face recognition refers to the facial image of the target task as the retrieval object, and searching for images matching the retrieval object in the database (for image retrieval, the database can also be called a gallery, a base library, a sample library, or a library to be retrieved). Image infringement judgment refers to the search service of copyright judgment, using images as the retrieval object, searching for the same or similar images in a massive gallery, and then determining whether there is an infringement problem with the retrieval object. Product recommendation refers to the search in the gallery based on the image provided by the user in the product image search function, finding the same or similar products, and then recommending the found products to the user. For the convenience of description, the following embodiments are explained by taking image retrieval as an example.

[0051] Image retrieval refers to the process of searching for images in a gallery that match the query image entered by the user. Generally, image retrieval is based on deep learning and image recognition technology, combined with different application businesses and industry scenarios, and uses feature vectorization and search capabilities to search for identical or similar images from a specified gallery. Specifically, image retrieval can usually include two steps:

[0052] Step one, use a feature extraction model (for example, a deep learning feature extraction model ResNet) to convert the image into a vector. In order to make the converted vector carry more information about the image, the converted vector is usually a high-dimensional vector, also referred to as a high-dimensional vector, such as a 256-dimensional vector or a 512-dimensional vector. The solution provided in the embodiment of the present application can use any feature extraction model to convert the image into a high-dimensional vector, and the embodiment of the present application does not limit this. Considering that in order to improve the subsequent retrieval speed, the high-dimensional vectors after the image in the base library is converted will be stored in the memory in a certain arrangement format in advance to form a massive image feature library.

[0053] Step 2: Use a specific distance method, such as cosine distance, L2 distance, Hamming distance, etc. to measure the similarity between two vectors. For example, when a user inputs an image to be retrieved, the system obtains the Hamming distance between the vector corresponding to the image in the gallery and the vector corresponding to the image to be retrieved, and outputs at least one image in the gallery or a sequence number corresponding to at least one image according to the order of the Hamming distance.

[0054] The following is an example of how to use the Hamming distance to perform image retrieval. Figure 1 , is a flow chart of the process of image retrieval using the Hamming distance. Feature extraction is performed on each image in the base library through the feature extraction model to convert each image in the base library into a binary feature. The embodiment of the present application also refers to the binary feature as a binary vector. The binary vector refers to a vector whose value of each dimension is 0 or 1. After the image to be retrieved is obtained, the same feature extraction model is used to extract the image to be retrieved to convert the image to be retrieved into a binary feature. The Hamming distance between the binary vector of the image to be retrieved and the binary vector of each image in the base library is calculated. Among them, the Hamming distance between two vectors refers to the number of difference bits between the two vectors. For example, the third and fifth bits of the vector (1,0,1,1,1,0,1) and the vector (1,0,0,1,0,0,1) are different, and the Hamming distance between the vector (1,0,1,1,1,0,1) and the vector (1,0,0,1,0,0,1) is 2. The obtained multiple Hamming distances are sorted to output at least one image in the gallery or a sequence number corresponding to at least one image. For example, the sequence numbers of the vectors corresponding to the images in the gallery with the smallest Hamming distances between K vectors corresponding to the image to be retrieved are output, where K is a positive integer.

[0055] Since the Hamming distance between two high-dimensional vectors is the number of their difference bits, on a system using a CPU for calculation, the XOR instruction is used to calculate the number of difference bits between the two high-dimensional vectors, and then the number of bits that are 1 is counted based on the result of the XOR calculation. This result is the Hamming distance between the two vectors. Specifically, each vector can be represented by a byte array. First, the XOR instruction is used to traverse each byte to obtain the XOR result, and then the popcnt instruction is used to count the number of bits in each byte that are 1. Finally, the binary heap sort algorithm is used to find the K candidate vectors with the closest Hamming distance.

[0056] The above process has at least the following defects:

[0057] (i) The entire process of using the Hamming distance for image retrieval mentioned above uses CPU resources for calculation. Specifically, when calculating the Hamming distance between two vectors through a program, it is necessary to use CPU resources to perform XOR calculations on each bit of the two vectors in turn, and add the results of the XOR calculations to obtain the Hamming distance between the two vectors. In the case where the number of bits of the vector is very large, multiple cycles of calculations are required during the above calculation process, and a large amount of data is transmitted between the memory and the CPU, resulting in a huge amount of calculations and a long calculation time. In the scenario where a large number of base images need to be queried, the amount of calculations is huge and the query performance is low.

[0058] (ii) Both the Hamming distance and the binary heap sort are calculated using CPU resources, so the process of calculating the Hamming distance and the process of sorting the Hamming distance must be executed in a serial manner, and asynchronous pipeline cannot be formed, and the image retrieval performance cannot be improved.

[0059] (III) In the process of obtaining the Hamming distance, it is necessary to rely on the popcnt instruction provided by the CPU platform. Some CPU platforms do not support the popcnt instruction and need to be implemented in other ways, which consumes more CPU resources and affects the performance of image retrieval.

[0060] In order to solve the above defects, the present application embodiment provides an image retrieval method, and the specific research ideas are as follows:

[0061] In view of defect (1), the solution provided in the embodiment of the present application converts the process of calculating the Hamming distance into a matrix multiplication operation, so the process of calculating the Hamming distance can be completed by a matrix multiplication calculation unit. This can reduce the amount of calculation required for calculating the Hamming distance, reduce the time required for calculating the Hamming distance, and improve the efficiency of image retrieval.

[0062] In response to defect (2), the solution provided in the embodiment of the present application divides the vectors corresponding to the images in the base library into multiple groups, completes the process of calculating the Hamming distance in the current group through a matrix multiplication calculation unit, and completes the process of sorting the Hamming distance obtained in the previous group through a vector calculation unit, thereby realizing asynchronous pipelining and improving image retrieval performance.

[0063] With respect to defect (three), the solution provided in the embodiment of the present application completes the sorting of the Hamming distances through a vector calculation unit, without relying on the popcnt instruction to complete the sorting of the Hamming distances, thereby improving the efficiency of image retrieval.

[0064] Based on the above research ideas, the solution provided by the embodiment of the present application is introduced below. In order to better understand the solution provided by the embodiment of the present application, the image retrieval system to which the method provided by the embodiment of the present application is applicable is first introduced below.

[0065] See also Figure 2 , is a schematic diagram of the architecture of the image retrieval system provided in an embodiment of the present application. As shown in the figure, the image retrieval system 100 includes a host system 10 and a retrieval system 20, wherein the host system 10 and the retrieval system 20 are connected via an interface bus 30. The interface bus 30 includes a peripheral component interconnect express (PCIe) or other types of interface buses.

[0066] The host system 10 is used to provide an application programming interface (API) or an image retrieval framework to the application running in the host system 10 or other systems, devices or applications outside the host system 10. The API or image retrieval framework is used to receive a user's data retrieval request (for image retrieval, the data retrieval request is an image retrieval request), and then send the data retrieval request to the retrieval system 20 through the interface bus 30, and the retrieval system 20 completes the processing of the data retrieval request. Optionally, the host system 10 is also used to run an operating system. The host system 10 includes a processor 101, which can be a central processing unit (CPU).

[0067] The retrieval system 20 is used to execute the retrieval request sent by the host system 10, and retrieve the data that is the same or similar to the query data in the database. The retrieval system 20 includes a matrix calculation unit 201, a vector calculation unit 202 and a main control logic unit 203. Among them, the matrix calculation unit 201, the vector calculation unit 202 and the main control logic unit 203 are connected through an internal bus, and the internal bus is a parallel bus for realizing point-to-point (P2P) communication. For example, the internal bus includes PCIe or front side bus (FSB) or other forms that can realize the connection of each component. The main control logic unit 203 is used to control the logic of the entire calculation process, send instructions to the matrix calculation unit 201 and the vector calculation unit 202, such as sending tasks to the matrix calculation unit 201 and the vector calculation unit 202, and synchronize the execution steps of the matrix calculation unit 201 and the vector calculation unit 202. The matrix calculation unit 201 is used to perform matrix-related operations. For example, the scheme provided in the embodiment of the present application can use matrix multiplication to obtain the Hamming distance between two vectors. The vector calculation unit 202 is used to perform vector related operations, for example, by copying the serial number corresponding to the Hamming distance to the specified position of the target matrix to sort multiple Hamming distances. The specific execution steps of the main control logic unit 203, the matrix calculation unit 201 and the vector calculation unit 202 will be introduced below, and will not be introduced here.

[0068] As a possible embodiment, the matrix calculation unit 201 and the vector calculation unit 202 can be respectively implemented by a hardware circuit with a specific function, which is used to perform calculation processing on data. For example, the matrix calculation unit 201 and the vector calculation unit 202 are respectively composed of a chip or a circuit structure constructed by electronic components. This embodiment is illustrated by taking the matrix calculation unit 201 as an example. The matrix calculation unit 201 has the versatility and programmability of a CPU, but is more specialized and can efficiently perform related operations, such as matrix-related operations. The matrix calculation unit 201 can be replaced by a processing chip such as a graphics processing unit (GPU) and an embedded neural network processor (neural-network processing units, NPU). Optionally, the matrix calculation unit 201 and the vector calculation unit 202 can also be implemented by software modules, or a mixture of software and hardware. Optionally, the matrix calculation unit 201 and the vector calculation unit 202 can also be deployed in one or partially in one, for example, the matrix calculation unit 201 and the vector calculation unit 202 are deployed in one, for implementing matrix-type operations and vector-type operations.

[0069] As a possible implementation, the retrieval system 20 may be an artificial intelligence (AI) chip, which includes a hardware circuit that implements the functions of the matrix calculation unit 201 and the vector calculation unit 202 .

[0070] As a possible embodiment, Figure 2 The retrieval system 20 shown can also be implemented in software. For example, the retrieval system 20 is a virtualized system, in which the matrix calculation unit 201 and the vector calculation unit 202 can be implemented by virtualized computing devices such as virtual machines or containers. At this time, the retrieval system 20 is connected to the host system 10 through a network, and the network can be in the form of Ethernet, optical fiber network, wireless network, etc. to realize the communication between the host system and the heterogeneous system. The virtualization system includes one or more devices, each of which includes a logic circuit or device for implementing the corresponding computing function. For example, the device may include the above-mentioned artificial intelligence chip. At this time, the matrix calculation unit 201 and the vector calculation unit 202 are implemented by one or more virtual machines or containers obtained after the device with the above-mentioned artificial intelligence chip is virtualized. For example, the virtualization system includes a device with a device (for example, an AI chip) that implements the function of a matrix computing engine, and multiple virtual machines or containers with virtual processors are obtained by virtualizing the resources of the device, and each virtual machine or container is used to implement the functions of the above-mentioned matrix calculation unit 201 and the vector calculation unit 202.

[0071] The following is an introduction to the image retrieval method provided in the embodiment of the present application. The image retrieval method provided in the embodiment of the present application can be applied to the above Figure 2 The image retrieval system described.

[0072] See also Figure 3 , which is a flow chart of an image retrieval method provided in an embodiment of the present application.

[0073] like Figure 3 As shown, an image retrieval method provided in an embodiment of the present application may include the following steps:

[0074] 301. A vector calculation unit obtains a query vector and multiple reference vectors.

[0075] The query vector is obtained based on the image to be retrieved, and the reference vector is obtained based on the image included in the image library. The reference vector can be understood as the vector included in the image feature library described above.

[0076] In one possible implementation, the query vector is a vector output by the feature extraction model after the image to be retrieved is input into the feature extraction model. The reference vector is a vector output by the feature extraction model after the image included in the gallery is input into the feature extraction model. The embodiment of the present application does not limit the specific type of feature extraction model, and any model that can convert an image into a vector representation can be adopted in the embodiment of the present application.

[0077] In one possible implementation, the query vector and the multiple reference vectors may be stored in a memory storage device, and the vector calculation unit may obtain the query vector and the multiple reference vectors from the memory storage device. The memory storage device may be a random access memory (RAM), a read only memory (ROM), a complementary metal oxide semiconductor memory (CMOS), etc., and the embodiments of the present application are not limited thereto.

[0078] 302. The vector calculation unit constructs a left matrix according to the query vector, and constructs a right matrix according to multiple reference vectors.

[0079] Assume that the query vector and the reference vector are both N-dimensional vectors, where N is a positive integer. There are M query vectors and P reference vectors in total. The vector calculation unit can construct an M-row N-column matrix based on the query vector, use it as the left matrix, construct a P-row N-column matrix based on the reference vector, transpose the P-row N-column matrix, and use the obtained N-row P-column matrix as the right matrix.

[0080] In one possible implementation, the query vector and the reference vector are both binary vectors, and the value of each dimension of the binary vector is 0 or 1. The value of each element in the left matrix is ​​-a or a, where a is a positive integer, and the position of the element with a value of -a in the left matrix is ​​obtained based on the position of the dimension with a value of 0 in the query vector. In other words, the position of the element with a value in the left matrix is ​​obtained based on the position of the element with a value in the query vector. The position of the element with a value in the left matrix is ​​obtained based on the position of the dimension with a value of 1 in the query vector. In other words, the position of the element with a value in the left matrix is ​​obtained based on the position of the element with a value in the query vector. Each element in the right matrix has a value of -a or a, wherein the position of the element with a value in the right matrix is ​​obtained based on the position of the dimension with a value of 0 in each reference vector. In other words, the position of the element with a value in the right matrix is ​​obtained based on the position of the element with a value in the reference vector. The position of the element with a value in the right matrix is ​​obtained based on the position of the dimension with a value of 1 in each reference vector. In other words, the position of the element with the value a in the right matrix is ​​obtained based on the position of the element with the value 1 in the reference vector. In order to better understand this implementation, a specific example is provided below:

[0081] To simplify the description, assume that the query vector and the reference vector are both four-dimensional vectors, that is, N is 4. And assume that the query vector includes two, (1,1,0,1) and (0,1,1,0), then the left matrix constructed based on the two query vectors is: The values ​​of the elements in the first row and first column of the left matrix are determined according to the values ​​of the first dimension of the first query vector. Since the value of the first dimension of the first query vector is 1, in this embodiment, the values ​​of the elements in the first row and first column are a. The values ​​of the elements in the first row and second column of the left matrix are determined according to the values ​​of the second dimension of the first query vector. Since the value of the second dimension of the first query vector is 1, in this embodiment, the values ​​of the elements in the first row and second column are a; the values ​​of the elements in the first row and third column of the left matrix are determined according to the values ​​of the third dimension of the first query vector. Since the value of the third dimension of the first query vector is 0, in this embodiment, the values ​​of the elements in the first row and second column are -a; the values ​​of the elements in the first row and fourth column of the left matrix are determined according to the values ​​of the fourth dimension of the first query vector. Since the value of the fourth dimension of the first query vector is 1, in this embodiment, the values ​​of the elements in the first row and second column are a. Similarly, the value of the element in the second row and first column of the left matrix is ​​determined according to the value of the first dimension of the second query vector. Since the value of the first dimension of the second query vector is 0, in this embodiment, the value of the element in the second row and first column is -a; the value of the element in the second row and second column of the left matrix is ​​determined according to the value of the second dimension of the second query vector. Since the value of the second dimension of the second query vector is 1, in this embodiment, the value of the element in the second row and first column is a; the value of the element in the second row and third column of the left matrix is ​​determined according to the value of the third dimension of the second query vector. Since the value of the third dimension of the second query vector is 1, in this embodiment, the value of the element in the second row and first column is a; the value of the element in the second row and fourth column of the left matrix is ​​determined according to the value of the fourth dimension of the second query vector. Since the value of the fourth dimension of the second query vector is 0, in this embodiment, the value of the element in the second row and first column is -a.

[0082] In a possible implementation, the value of a is 1. Taking the value of a being 1 as an example, a possible method of constructing the left matrix and the right matrix is ​​given below. Figure 4, pre-store N-dimensional all-1 vectors and N-dimensional all-1 vectors in the memory storage. Among them, the N-dimensional all-1 vector means that the value of each dimension of the N-dimensional vector is 1, and the N-dimensional all-1 vector means that the value of each dimension of the N-dimensional vector is -1. Traverse the value of each dimension of each query vector, and obtain the value of a row of elements in the left matrix according to a query vector. Specifically, if the value of a certain dimension of the query vector traversed is 1, the vector selection instruction is executed by the vector calculation unit, and the corresponding value is selected from the corresponding dimension of the N-dimensional all-1 vector and added to the corresponding position of the left matrix. For example, if the value of the first dimension of the first query vector traversed is 1, the vector selection instruction is executed by the vector calculation unit, and the corresponding value 1 is selected from the first dimension of the N-dimensional all-1 vector and added to the first row and first column of the left matrix. If the value of a certain dimension of the query vector traversed is 0, the vector selection instruction is executed by the vector calculation unit, and the corresponding value is selected from the corresponding dimension of the N-dimensional all-1 vector and added to the corresponding position of the left matrix. For example, if the value of the second dimension of the first query vector is 0, the vector selection instruction is executed through the vector calculation unit, and the corresponding value -1 is selected from the second dimension of the N-dimensional all-1 vector and added to the first row and second column of the left matrix. Similarly, after traversing all the dimensions of all query vectors, the construction of the left matrix can be completed. In the process of constructing the right matrix, the value of each dimension of each reference vector is traversed, and the value of a column of elements in the right matrix is ​​obtained according to a reference vector. Specifically, if the value of a certain dimension of the reference vector is 1, the vector selection instruction is executed through the vector calculation unit, and the corresponding value is selected from the corresponding dimension of the N-dimensional all-1 vector and added to the corresponding position of the first matrix. For example, if the value of the first dimension of the first reference vector is 1, the vector selection instruction is executed through the vector calculation unit, and the corresponding value 1 is selected from the first dimension of the N-dimensional all-1 vector and added to the first row and first column of the first matrix. When the value of a certain dimension of the reference vector is 0, the vector selection instruction is executed through the vector calculation unit to select the corresponding value from the corresponding dimension of the N-dimensional all-1 vector and add it to the corresponding position of the first matrix. Similarly, after traversing all dimensions of all reference vectors, the construction of the first matrix can be completed, and then the first matrix is ​​transposed to complete the construction of the right matrix.

[0083] 303. The matrix calculation unit performs matrix multiplication calculation on the left matrix and the right matrix to obtain the Hamming distance between the query vector and each reference vector.

[0084] The solution provided in the embodiment of the present application transforms the process of calculating the Hamming distance into the process of calculating matrix multiplication, so the process of calculating the Hamming distance can be completed by the matrix calculation unit. The following two specific examples illustrate how to transform the process of calculating the Hamming distance into the process of matrix multiplication.

[0085] Since the value of each element in the left matrix and the value of each element in the right matrix are both a or -a, when performing matrix multiplication on the left matrix and the right matrix, if the values ​​of two elements are the same, the product of the two is a. 2 , if the values ​​of two elements are different, then the product of the two is (-a) 2 For the sake of convenience, assume that the query vector 1 is (1, 1, 0, 1) and the reference vector 1 is (1, 0, 1, 1). Then the left matrix is ​​[aa-aa] and the right matrix is

[0086] For the same bits of query vector 1 and reference vector 1, when performing matrix multiplication, the product of the elements corresponding to the same bits is a 2 , for the difference between query vector 1 and reference vector 1, when performing matrix multiplication, the result of the element corresponding to the difference is (-a) 2 . Specifically:

[0087] The first bit of query vector 1 is 1, and the first bit of reference vector 1 is also 1. They are the same bit. When performing matrix multiplication, the first bit of query vector 1 corresponds to the element of the first row and first column of the left matrix, and its value is a. The first bit of reference vector 1 corresponds to the element of the first row and first column of the right matrix, and its value is also a. Then the product of the element of the first row and first column of the left matrix and the element of the first row and first column of the right matrix is ​​a. 2 .

[0088] The second bit of query vector 1 is 1, and the second bit of reference vector is 0. The two are difference bits. When performing matrix multiplication, the second bit of query vector 1 corresponds to the element of the first row and second column of the left matrix, and its value is a. The second bit of reference vector 1 corresponds to the element of the first row and second column of the right matrix, and its value is -a. Then the product of the element of the first row and second column of the left matrix and the element of the first row and second column of the right matrix is ​​(-a). 2 .

[0089] The third bit of query vector 1 is 0, and the third bit of reference vector is 1. The two are different bits. When performing matrix multiplication, the third bit of query vector 1 corresponds to the element of the first row and third column of the left matrix, and its value is -a. The third bit of reference vector 1 corresponds to the element of the first row and third column of the right matrix, and its value is a. Then the product of the element of the first row and third column of the left matrix and the element of the first row and third column of the right matrix is ​​(-a). 2 .

[0090] The fourth bit of query vector 1 is 1, and the fourth bit of reference vector 1 is also 1. They are the same bit. When performing matrix multiplication, the fourth bit of query vector 1 corresponds to the element in the first row and fourth column of the left matrix, and its value is a. The fourth bit of reference vector 1 corresponds to the element in the first row and fourth column of the right matrix, and its value is also a. Then the product of the element in the first row and fourth column of the left matrix and the element in the first row and fourth column of the right matrix is ​​a. 2 .

[0091] In summary, assuming that the result of multiplying the left matrix and the right matrix is ​​cal, then cal = a 2 +(-a) 2 +(-a) 2 +a 2 As mentioned above, calculating the Hamming distance between two vectors is to calculate the number of different bits between the two vectors. Here, assuming that the number of different bits between a query vector and a reference vector is x, and the number of identical bits is y, then calculating the Hamming distance between the query vector and the reference vector is to calculate the value of x. Then the above formula cal = a 2 +(-a) 2 +(-a) 2 +a 2 Can be converted to cal = -xa 2 +ya 2 In addition, if both the query vector and the reference vector are N-dimensional vectors, then the formula N=x+y is valid. Based on the above two formulas:

[0092]

[0093] Since a is a known positive integer, cal is the product of a row of the left matrix and a column of the right matrix, and N is the known dimension of the query vector and the reference vector, the value of x can be obtained according to the above two formulas, that is, the Hamming distance between a query vector and a reference vector. When the value of a is 1, the above formula can be converted to:

[0094]

[0095] The Hamming distance between a query vector and a reference vector can be obtained based on the product of a row of the left matrix and a column of the right matrix and N, that is,

[0096] The above describes how to obtain the Hamming distance between a query vector and a reference vector. Similarly, continue to refer to Figure 4 When there are multiple query vectors and multiple reference vectors, referring to the above process, a matrix of M rows and P columns can be obtained according to the product of the left matrix and the right matrix. According to the value of each element in the matrix of M rows and P columns and N, the Hamming distance between each query vector and each reference vector can be obtained by referring to the above formula.

[0097] 304. The vector calculation unit obtains a search result according to the sorting results of the multiple Hamming distances.

[0098] The search results include images in the gallery that are similar to the image to be searched.

[0099] In a possible implementation, the multiple Hamming distances may be sorted in ascending order or in descending order, and the serial numbers of the K reference vectors with the smallest Hamming distances to the query vector may be output, or the images in the gallery corresponding to the K reference vectors with the smallest Hamming distances to the query vector may be output.

[0100] The solution provided in the embodiment of the present application converts the process of calculating the Hamming distance into a matrix multiplication operation, so the process of calculating the Hamming distance can be completed by a matrix multiplication calculation unit. The process of calculating the Hamming distance by the matrix multiplication calculation unit does not need to perform cumbersome processes, such as there is no need to perform XOR calculations on each bit of the two vectors in turn, and add the results of the XOR calculations to obtain the Hamming distance between the two vectors. The matrix multiplication calculation unit can be specifically used to perform matrix multiplication calculations. Compared with using CPU resources to calculate the Hamming distance, calculating the Hamming distance by the matrix multiplication calculation unit can improve the calculation efficiency. Thereby, the amount of calculation required to calculate the Hamming distance between the query vector and the reference vector during the image retrieval process can be reduced, the time required to calculate the Hamming distance between the query vector and the reference vector can be reduced, and the efficiency of image retrieval can be improved.

[0101] In a possible implementation, a scatter instruction can be executed by a vector computing unit to complete the sorting results of multiple Hamming distances, which can improve the efficiency of sorting multiple Hamming distances and further improve the efficiency of image retrieval, which is introduced below in conjunction with a specific embodiment.

[0102] See also Figure 5 , which is a flow chart of an image retrieval method provided in an embodiment of the present application.

[0103] like Figure 5 As shown, an image retrieval method provided in an embodiment of the present application may include the following steps:

[0104] 501. A vector calculation unit obtains a query vector and multiple reference vectors.

[0105] 502. The vector calculation unit constructs a left matrix according to the query vector, and constructs a right matrix according to multiple reference vectors.

[0106] 503. The matrix calculation unit performs matrix multiplication calculation on the left matrix and the right matrix to obtain the Hamming distance between the query vector and each reference vector.

[0107] Steps 501 to 503 can refer to Figure 3 Steps 301 to 303 in the corresponding embodiment are understood and will not be repeated here.

[0108] 504. The vector calculation unit copies the serial numbers corresponding to the multiple Hamming distances to the target matrix of N+1 rows and K columns.

[0109] K is a positive integer. The values ​​of the Hamming distances corresponding to the serial numbers stored in each row of the target matrix are the same. The value of the Hamming distance corresponding to any serial number stored in the previous row in N+1 rows is smaller than the value of the Hamming distance corresponding to any serial number stored in the next row.

[0110] Since the query vector and the reference vector are both N-dimensional vectors, the Hamming distance between the query vector and the reference vector must be one of the (N+1) values ​​between 0 and N. For example, if the query vector and the reference vector are both 256-dimensional vectors, the minimum Hamming distance between a query vector and a reference vector may be 0 at the minimum and 256 at the maximum, and the Hamming distance between the query vector and the reference vector may be any value between 0 and 256. Therefore, the number of rows in the target matrix is ​​N+1, and the Hamming distances corresponding to the sequence numbers stored in each row are the same. And as the number of rows in the N+1 row increases, the Hamming distances corresponding to the stored sequence numbers increase. For example, the Hamming distances corresponding to the sequence numbers stored in the first row of the N+1 row are all 0, the Hamming distances corresponding to the sequence numbers stored in the second row of the N+1 row are all 1, the Hamming distances corresponding to the sequence numbers stored in the third row of the N+1 row are all 2, and the Hamming distances corresponding to the sequence numbers stored in the fourth row of the N+1 row are 3, and so on.

[0111] It can be considered that each image in the image library has its own serial number, or that each reference vector in the image feature library has its own serial number. Then the serial number of the Hamming distance between the reference vector and a query vector is the same as the serial number of the image, or the same as the serial number of the reference vector.

[0112] Assuming that in one image search, a total of K search results are to be obtained, the number of columns of the target matrix is ​​K. Therefore, the serial numbers stored in the first row of the target matrix can be directly read to obtain the K search results. If the serial numbers stored in the first row of the target matrix are less than K, the serial numbers stored in the next row can be read continuously until K serial numbers are read.

[0113] In a possible implementation, if the number of identical Hamming distances exceeds K, the sequence numbers stored in the corresponding rows can be replaced by the sequence numbers with the identical Hamming distances obtained later. For example, the Hamming distances corresponding to the sequence numbers stored in the first row are all 0, and a total of K sequence numbers are currently stored, and the Hamming distances corresponding to the K sequence numbers are all 0. When the Hamming distance corresponding to the K+1th sequence number is also 0, the K+1th sequence number can be used to replace a sequence number in the first row, for example, the first sequence number in the first row can be replaced.

[0114] In one possible implementation, multiple Hamming distances can be traversed to count the number of current target Hamming distances, where the target Hamming distance is any one of the multiple Hamming distances. The offset vector of the target Hamming distance is obtained based on the number of current target Hamming distances, the value of the target Hamming distance, and K. The serial number corresponding to the target Hamming distance is copied to a specified position of the target matrix, where the specified position is obtained based on the offset vector. In order to better understand this implementation, a specific example is provided below for explanation:

[0115] See also Figure 6 , traverse multiple Hamming distances, and in the process of traversal, count the number of corresponding values ​​of each current Hamming distance, or count the number of reference vectors corresponding to each current Hamming distance value. For example, Figure 6 In the example shown, when the first Hamming distance is traversed, the value of the first Hamming distance is 1, and the number of Hamming distances with a current value of 1 is 1. Assuming that when the second Hamming distance is traversed, the value of the second Hamming distance is also 1, when the second Hamming distance is traversed, the number of Hamming distances with a current value of 1 is 2. According to the number of the current Hamming distance, the value of the Hamming distance, and K, the offset vector of the Hamming distance is obtained. In a possible implementation, the formula d n *K+cnt[d n ] to understand. Among them, d n is the value of the Hamming distance, cnt[d n ] is the current Hamming distance value d n For example, in Figure 6In the example shown, the value of the first Hamming distance among the multiple Hamming distances is 1, and the sequence number corresponding to the first Hamming distance is 100. Since it is the first Hamming distance, the number of Hamming distances currently counted with a value of 1 is 1, so cnt[d n ] is 1, then refer to formula d n *K+cnt[d n ], the offset vector of the first Hamming distance is 1*K+1, then the serial number 100 is copied to the 1*K+1 position of the target matrix, that is, the position of the second row and the first column of the target matrix. For another example, the value of the fifth Hamming distance among the multiple Hamming distances is 3, and the serial number corresponding to the fifth Hamming distance is 104. Assuming that the number of Hamming distances with a value of 3 currently counted is 2, refer to formula d n *K+cnt[d n ], the offset vector of the fifth Hamming distance is obtained as 3*K+2, then the serial number 104 is copied to the 3*K+2th position of the target matrix, that is, the fourth row and second column of the target matrix.

[0116] In one possible implementation, a scatter instruction may be called to copy the sequence number corresponding to the target Hamming distance to a specified position of the target matrix. In this implementation, a scatter instruction may be called to perform a copy operation to copy a segment of continuous sequence numbers to a specified position in the target matrix in a scattered manner. The process of determining the specified position based on the offset vector has been illustrated in the previous paragraph and will not be repeated here.

[0117] 505. The main control logic unit obtains K serial numbers according to the target matrix, and the K serial numbers are used to obtain the search results.

[0118] The solution provided by the embodiment of the present application can copy the serial numbers corresponding to multiple Hamming distances or the serial numbers corresponding to multiple reference vectors to the target matrix through the vector calculation unit, and the main control logic unit can directly read the serial numbers stored in the first row of the target matrix to obtain K search results. If the serial numbers stored in the first row of the target matrix are less than K, the main control logic unit can continue to read the serial numbers stored in the next row until K serial numbers are read. In this way, the sorting of the Hamming distances is completed by the vector calculation unit, and there is no need to rely on the popcnt instruction to complete the sorting of the Hamming distances, which reduces the utilization of CPU resources and can improve the efficiency of image retrieval.

[0119] In a possible implementation, the matrix calculation unit and the vector calculation unit can perform asynchronous pipeline execution, and end-to-end execute the solution provided by the embodiment of the present application to improve the query per second (QPS) performance. The following is an introduction in conjunction with a specific embodiment.

[0120] See also Figure 7 , which is a flow chart of an image retrieval method provided in an embodiment of the present application.

[0121] like Figure 7 As shown, an image retrieval method provided in an embodiment of the present application may include the following steps:

[0122] 701. The main control logic unit divides multiple reference vectors into multiple groups.

[0123] Each group includes at least one reference vector. For example, if there are 10 million reference vectors in the image feature library, the 10 million reference vectors can be divided into 10 groups, each group including 1 million reference vectors. Or it can be understood that the main control logic unit divides multiple reference vectors into multiple task blocks, and each task block processes at least one reference vector. For example, there are 10 million reference vectors in the image feature library, and each task processes 1 million, that is, 10 tasks are executed. In order to facilitate the explanation of the scheme, the embodiment of the present application takes the first group and the second group included in the multiple groups as an example to illustrate the asynchronous pipeline execution scheme of the matrix calculation unit and the vector calculation unit. Or it can be understood that the following takes two task blocks as an example to illustrate that the matrix calculation unit and the vector calculation unit can asynchronously pipeline process the two tasks.

[0124] 702. The vector calculation unit constructs a left matrix according to the query vector, and constructs a first right matrix according to the reference vector included in the first group.

[0125] Step 702 can refer to Figure 3 The corresponding step 302 in the embodiment is understood and will not be repeated here. The difference is that Figure 3 In the corresponding embodiment, the right matrix can be constructed according to all reference vectors, Figure 7 In a corresponding embodiment, a first right matrix is ​​constructed according to the reference vectors included in the first group.

[0126] 703. The matrix calculation unit performs matrix multiplication calculation on the left matrix and the first right matrix to obtain the Hamming distance between the query vector and each reference vector included in the first group.

[0127] Step 703 can refer to Figure 3 The step 303 in the corresponding embodiment is understood and will not be repeated here. The difference is that Figure 3 In the corresponding embodiment, matrix multiplication is performed on the left matrix and the right matrix. Figure 7 In a corresponding embodiment, matrix multiplication is performed on the left matrix and the first right matrix.

[0128] 704. The vector calculation unit copies the sequence numbers corresponding to the multiple Hamming distances of the first group to the target matrix of N+1 rows and K columns.

[0129] Step 704 can refer to Figure 5 The corresponding step 504 in the embodiment is understood and will not be repeated here. The difference is that Figure 5 In a corresponding embodiment, the vector calculation unit may copy the serial numbers corresponding to all the multiple Hamming distances to the target matrix of N+1 rows and K columns. Figure 7 In a corresponding embodiment, the vector calculation unit copies the sequence numbers corresponding to the multiple Hamming distances of the first group to the target matrix of N+1 rows and K columns.

[0130] 705. The vector calculation unit constructs a second right matrix according to the reference vectors included in the second group.

[0131] Step 705 and step 703 may be performed simultaneously.

[0132] Step 705 can refer to Figure 3 The corresponding step 302 in the embodiment is understood and will not be repeated here. The difference is that Figure 3 In the corresponding embodiment, the right matrix can be constructed according to all reference vectors, Figure 7 In a corresponding embodiment, a second right matrix is ​​constructed according to the reference vectors included in the second group.

[0133] 706. The matrix calculation unit performs matrix multiplication calculation on the left matrix and the second right matrix to obtain the Hamming distance between the query vector and each reference vector included in the second group.

[0134] Step 706 and step 704 can be performed simultaneously. It can also be understood that when obtaining the Hamming distance between the query vector and each reference vector in the second group, the sequence number corresponding to the Hamming distance between the query vector and each reference vector in the first group is copied to the target matrix with N+1 rows and K columns.

[0135] Step 706 can refer to Figure 3 The step 303 in the corresponding embodiment is understood and will not be repeated here. The difference is that Figure 3 In the corresponding embodiment, matrix multiplication is performed on the left matrix and the right matrix. Figure 7 In a corresponding embodiment, matrix multiplication is performed on the left matrix and the second right matrix.

[0136] 707. The vector calculation unit copies the sequence numbers corresponding to the plurality of Hamming distances of the second group to the target matrix of N+1 rows and K columns.

[0137] Step 707 can refer to Figure 5The corresponding step 504 in the embodiment is understood and will not be repeated here. The difference is that Figure 5 In a corresponding embodiment, the vector calculation unit may copy the serial numbers corresponding to all the multiple Hamming distances to the target matrix of N+1 rows and K columns. Figure 7 In a corresponding embodiment, the vector calculation unit copies the sequence numbers corresponding to the plurality of Hamming distances of the second group to the target matrix of N+1 rows and K columns.

[0138] According to the process of executing the first group and the second group described above, each group (or each task block) can be executed, and then the Hamming distance between the query vector and each reference vector can be obtained. After each group is executed, the vector calculation unit updates the target matrix. Finally, all groups are executed, and the target matrix includes the serial number corresponding to each Hamming distance.

[0139] 708. The main control logic unit obtains K serial numbers according to the target matrix, and the K serial numbers are used to obtain the search results.

[0140] Step 708 can refer to Figure 5 The corresponding step 505 in the embodiment can be understood and will not be repeated here.

[0141] According to the solution provided in the embodiment of the present application, the main control logic unit can send tasks to the vector calculation unit and the matrix calculation unit at the same time at the granularity of tasks, or at the granularity of groups, so as to increase the resource utilization of the vector calculation unit and the matrix calculation unit.

[0142] See also Figure 8 , the solution provided in the embodiment of the present application mainly involves the process of calculating the Hamming distance in the image retrieval process and the process of sorting multiple Hamming distances, which improves the overall performance of the image retrieval system. Specifically, the solution provided in the embodiment of the present application uses matrix multiplication operations instead of XOR calculations to obtain the Hamming distance between vectors. The solution provided in the embodiment of the present application can use a vector Scatter copy method instead of a binary heap algorithm to sort multiple Hamming distances. The specifics can be understood by referring to the various implementation methods described above.

[0143] In order to better understand the solution provided in the embodiment of the present application, the timing diagram involved in the solution provided in the embodiment of the present application is introduced below.

[0144] See also Fig. 9 , is a timing flow chart of the image retrieval method provided in the embodiment of the present application. Fig. 9 As shown, the main control logic unit divides the multiple reference vectors of the image feature library into task blocks, which can be Figure 7The step 701 in the corresponding embodiment can be understood. Next, the main control logic unit determines whether each task block has been executed. If there are still task blocks that have not been executed, the main control logic unit starts the task blocks that have not been executed. Specifically, the main control logic unit can send the starting address of the reference vector included in the task block that has not been executed, and the number of reference vectors included in the task block to the vector calculation unit. The vector calculation unit obtains a set of reference vectors from the memory storage according to the obtained starting address of the reference vector and the number of reference vectors included in the task block. Then, the vector calculation unit constructs a right matrix based on the obtained reference vector and a left matrix based on the obtained query vector, which can be referred to. Figure 7 The process of constructing the first right matrix and the process of constructing the second right matrix in the corresponding embodiment are understood. Next, the matrix calculation unit performs matrix multiplication on the left matrix and the right matrix to obtain the matrix multiplication result. The vector calculation unit obtains multiple Hamming distances according to each element in the matrix multiplication result and N. The main control logic unit returns the task and outputs the Hamming distance between each reference vector and the query vector included in the current task module. If all task blocks are executed, the process ends, and the end of the process represents that the Hamming distance between each reference vector and the query vector has been obtained.

[0145] Next, see Fig.10 , is another timing flow chart of the image retrieval method provided in the embodiment of the present application. Fig.10 As shown, first, the main control logic unit can initialize the data structure, such as initializing the target matrix. Then, the main control logic unit determines whether the Hamming distance obtained based on each task block has been traversed. If the Hamming distance obtained based on each task block has not been traversed, the task block that has not been traversed is obtained, and specifically the Hamming distance obtained based on the task block is obtained. The vector calculation unit obtains the offset vector based on the Hamming distance obtained by the task block, and performs a scatter copy operation based on the offset vector to update the target matrix. The main control logic unit updates the number of each Hamming distance, or updates the number of reference vectors corresponding to each Hamming distance, wherein the step of updating the number of reference vectors corresponding to each Hamming distance can also be performed at other times, such as when the main control logic unit obtains the Hamming distance obtained based on the task block. If the main control logic unit determines that the Hamming distance obtained based on each task block has been traversed, the process ends. The end of the process means that the target matrix can be output and K search results can be obtained through the target matrix.

[0146] The calculation of the Hamming distance can be completed by matrix multiplication, and the sorting of the Hamming distance can be completed by vector scatter copying. In one possible implementation, the solution provided in the embodiment of the present application can integrate the matrix calculation unit and the vector calculation unit together, and form an asynchronous pipeline mechanism, and finally realize the end-to-end full process based on Hamming distance retrieval. The total reference vector can be divided into several blocks according to a certain size and input into the matrix calculation unit respectively. After the Hamming distance calculation corresponding to the block is completed, it is input into the vector calculation unit for sorting calculation, and finally the target matrix is ​​updated. Since the block processing is adopted, and the matrix calculation unit and the vector calculation unit are two independent physical computing units, an asynchronous pipeline can be formed between them, such as Fig.11 As shown, from the time axis, the matrix calculation unit and the vector calculation unit can cover each other in part of the execution time.

[0147] As a possible embodiment, the above-mentioned image retrieval processing method can also be applied to video retrieval, wherein the retrieval process of each frame image in the video can refer to the above-mentioned Figures 3 to 7 The processing process of the method shown, and then determining the retrieval results of the video based on the retrieval results of each image, will not be repeated here for the sake of brevity.

[0148] It is worth noting that, for the above method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required for the present application.

[0149] Combined with the above Figures 2 to 11 , describes in detail the image retrieval method provided by the embodiment of the present application, and will be combined with Fig.12 , describing an image retrieval device provided according to an embodiment of the present application.

[0150] Fig.12 The schematic diagram of the structure of an image retrieval device 1200 provided in the present application is as shown in the figure, the device 1200 includes a vector calculation unit 1201, a matrix calculation unit 1202 and a memory storage 1203. In a possible implementation, the device 1200 may also include a main control logic unit 1204.

[0151] It should be understood that the matrix calculation unit 1202 in the device 1200 of the embodiment of the present application may correspond to Figure 2 In the implementation of the matrix calculation unit 201, the vector calculation unit 1201 can correspond to Figure 2 The main control logic unit 1204 may correspond to Figure 2 Implementation of the main control logic unit 203.

[0152] The vector calculation unit 1201 can be used to sort the Hamming distances and perform scatter copy operations. Figures 3 to 7 The matrix calculation unit 1202 can be used to calculate the Hamming distance between binary vectors, specifically, it can be performed Figures 3 to 7 The main control logic unit 1204 can be used to control the logic of the entire calculation process, control the sending and synchronization of tasks to the matrix calculation unit and the vector calculation unit, and can be used to perform Figures 3 to 7 The related steps performed by the main control logic unit in the described implementation mode. The memory storage can be used to store the binary vector, and the matrix calculation unit and the vector calculation unit can both read the vector from the memory storage for processing.

[0153] In one possible implementation, the vector calculation unit 1201 is used to obtain a query vector and multiple reference vectors from the memory storage 1203, wherein the query vector is obtained based on the image to be retrieved, and the reference vector is obtained based on the images included in the gallery. The matrix calculation unit 1202 is used to perform matrix multiplication calculation on the left matrix constructed by the vector calculation unit 1201 based on the query vector and the right matrix constructed by the vector calculation unit 1201 based on the multiple reference vectors, so as to obtain the Hamming distance between the query vector and each reference vector. The vector calculation unit 1201 is also used to obtain a retrieval result based on the sorting results of the multiple Hamming distances, and the retrieval result includes images in the gallery that are similar to the image to be retrieved.

[0154] In a possible implementation, the query vector and the reference vector are both N-dimensional vectors, where N is a positive integer, and the matrix calculation unit 1202 is specifically configured to: perform matrix multiplication calculation on the left matrix and the right matrix to obtain a first calculation result. According to the first calculation result and N, the Hamming distance between the query vector and each reference vector is obtained.

[0155] In a possible implementation, the query vector and the reference vector are both binary vectors, and the value of each dimension of the binary vector is 0 or 1. The value of each element in the left matrix is ​​-a or a, where a is a positive integer. The position of the element with a value of -a in the left matrix is ​​obtained based on the position of the element with a value of 0 in the query vector, and the position of the element with a value in the left matrix is ​​obtained based on the position of the element with a value of 1 in the query vector. The value of each element in the right matrix is ​​-a or a, where the position of the element with a value of -a in the right matrix is ​​obtained based on the position of the element with a value of 0 in each reference vector, and the position of the element with a value in the right matrix is ​​obtained based on the position of the element with a value of 1 in each reference vector.

[0156] In a possible implementation manner, the value of a is 1.

[0157] In a possible implementation, the vector calculation unit 1201 is specifically used to: copy the serial numbers corresponding to the multiple Hamming distances to a target matrix of N+1 rows and K columns, K is a positive integer, the values ​​of the Hamming distances corresponding to the serial numbers stored in each row of the target matrix are the same, and the value of the Hamming distance corresponding to any serial number stored in the previous row of N+1 rows is less than the value of the Hamming distance corresponding to any serial number stored in the next row. K serial numbers are obtained according to the target matrix, and the K serial numbers are used to obtain the search results.

[0158] In a possible implementation, the vector calculation unit 1201 is specifically configured to: traverse multiple Hamming distances to count the number of current target Hamming distances, where the target Hamming distance is any one of the multiple Hamming distances.

[0159] The offset vector of the target Hamming distance is obtained according to the number of the current target Hamming distance, the value of the target Hamming distance, and K. The serial number corresponding to the target Hamming distance is copied to the specified position of the target matrix, and the specified position is obtained based on the offset vector.

[0160] In a possible implementation, the vector calculation unit 1201 is specifically configured to: call a scatter instruction to copy the sequence number corresponding to the target Hamming distance to a specified position of the target matrix.

[0161] In a possible implementation, the device further includes a main control logic unit 1204, the main control logic unit 1204 is used to divide the multiple reference vectors into multiple groups, each group includes at least one reference vector, and the multiple groups include a first group and a second group. The vector calculation unit 1201 is specifically used to copy the sequence number corresponding to the Hamming distance between the query vector and each reference vector in the first group to the target matrix of N+1 rows and K columns when the matrix calculation unit 1202 obtains the Hamming distance between the query vector and each reference vector in the second group.

[0162] In one possible implementation, the utilization rate of the matrix calculation unit 1202 exceeds 60%.

[0163] The matrix calculation unit 1202 and the vector calculation unit 1201 can also be implemented by software. Figures 3 to 7 When the image retrieval method shown in the figure is used, the device 1200 and its various modules may also be software modules. For example, the GPU calls the instructions in the memory to respectively implement the functions of the matrix calculation unit 1202 and the vector calculation unit 1201. Optionally, when the matrix calculation unit 1202 and the vector calculation unit 1201 are implemented by software, the CPU or other types of processors may also call the computer instructions in the memory to respectively implement the functions of the matrix calculation unit 1202 and the vector calculation unit 1201. Among them, other types of processors include digital signal processors (digital signal processing, DSP), application-specific integrated circuits (application-specific integrated circuits, ASIC), field-programmable gate arrays (field-programmable gate arrays, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0164] The device 1200 according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each unit in the device 1200 are respectively to implement Figures 3 to 7 For the sake of brevity, the corresponding processes of each method are not repeated here.

[0165] In a possible implementation, the image retrieval device may be an all-in-one retrieval machine (a hardware server and software bundled together) or a retrieval card (an acceleration card for image retrieval, generally inserted into a server PCIe slot for use).

[0166] The present application also provides an image retrieval system, which includes a matrix calculation unit and a vector calculation unit, which are connected via an internal bus, and the matrix calculation unit and the vector calculation unit are respectively used to implement the following Figures 3 to 7 The functions of the corresponding execution entities in the description will not be repeated here for the sake of brevity.

[0167] In a possible implementation, a software development kit (SDK) for image retrieval can be provided to the user at the software, so that the user can implement the image retrieval application. Alternatively, it can be understood that the image retrieval device provided in the embodiment of the present application can be located in the application of the local terminal, or can also be a plug-in or SDK and other functional units located in the application of the local terminal, or can also be located in the server, which is not particularly limited in the embodiment of the present invention.

[0168] The present application also provides a data processing system, the data processing system comprising: Figure 2 The host system and the retrieval system shown in the figure include a matrix calculation unit and a vector calculation unit, the matrix calculation unit and the vector calculation unit are connected through an internal bus, and the matrix calculation unit and the vector calculation unit are respectively used to implement the following Figures 3 to 7 The functions of the corresponding execution entities in the description will not be repeated here for the sake of brevity.

[0169] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0170] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

Claims

1. A method for image retrieval, characterized in that: include: Using a vector calculation unit to obtain a query vector and a plurality of reference vectors, wherein the query vector is obtained based on the image to be retrieved, and the reference vector is obtained based on the images included in the gallery; A matrix calculation unit is used to perform matrix multiplication on a left matrix constructed based on the query vector and a right matrix constructed based on the multiple reference vectors to obtain the Hamming distance between the query vector and each of the reference vectors; a retrieval result is obtained based on the sorting results of the multiple Hamming distances, and the retrieval result includes images in the gallery that are similar to the image to be retrieved.

2. The method according to claim 1, characterized in that The query vector and the reference vector are both N-dimensional vectors, where N is a positive integer, and performing matrix multiplication calculation on a left matrix constructed based on the query vector and a right matrix constructed based on the multiple reference vectors to obtain a Hamming distance between the query vector and each of the reference vectors includes: Performing matrix multiplication calculation on the left matrix and the right matrix to obtain a first calculation result; The Hamming distance between the query vector and each of the reference vectors is obtained according to the first calculation result and N.

3. The method according to claim 2, characterized in that The query vector and the reference vector are both binary vectors, and the value of each dimension of the binary vector is 0 or 1. The value of each element in the left matrix is ​​-a or a, where a is a positive integer. The position of the element with a value of -a in the left matrix is ​​obtained based on the position of the element with a value of 0 in the query vector, and the position of the element with a value in the left matrix is ​​obtained based on the position of the element with a value of 1 in the query vector. The value of each element in the right matrix is ​​-a or a, wherein the position of the element with a value of -a in the right matrix is ​​obtained based on the position of the element with a value of 0 in each of the reference vectors, and the position of the element with a value in the right matrix is ​​obtained based on the position of the element with a value of 1 in each of the reference vectors.

4. The method according to claim 3, characterized in that The value of a is 1.

5. The method according to any one of claims 2 to 4, characterized in that: The step of obtaining the search result according to the sorting results of the plurality of Hamming distances comprises: Copy the sequence numbers corresponding to the multiple Hamming distances to a target matrix of N+1 rows and K columns, where K is a positive integer, the values ​​of the Hamming distances corresponding to the sequence numbers stored in each row of the target matrix are the same, and the value of the Hamming distance corresponding to any sequence number stored in the previous row of the N+1 rows is less than the value of the Hamming distance corresponding to any sequence number stored in the next row; K serial numbers are obtained according to the target matrix, and the K serial numbers are used to obtain the search results.

6. The method according to claim 5, characterized in that The step of copying the sequence numbers corresponding to the plurality of Hamming distances to a target matrix having N+1 rows and K columns includes: Traversing the multiple Hamming distances to count the number of current target Hamming distances, where the target Hamming distance is any one of the multiple Hamming distances; Obtaining an offset vector of the target Hamming distance according to the current number of the target Hamming distances, the value of the target Hamming distance and K; The serial number corresponding to the target Hamming distance is copied to a specified position of the target matrix, where the specified position is obtained based on the offset vector.

7. The method according to claim 6, characterized in that The step of copying the sequence number corresponding to the target Hamming distance to a specified position of the target matrix includes: The scatter instruction is called to copy the serial number corresponding to the target Hamming distance to the specified position of the target matrix.

8. The method according to any one of claims 5 to 7, characterized in that: The method further comprises: Divide the plurality of reference vectors into a plurality of groups, each of the groups comprising at least one reference vector, and the plurality of groups comprising a first group and a second group; The step of copying the sequence numbers corresponding to the plurality of Hamming distances to a target matrix having N+1 rows and K columns includes: When obtaining the Hamming distance between the query vector and each of the reference vectors in the second group, the serial number corresponding to the Hamming distance between the query vector and each of the reference vectors in the first group is copied to the target matrix of N+1 rows and K columns.

9. An image retrieval device, characterized in that: The device comprises a vector calculation unit and a matrix calculation unit, The vector calculation unit is used to obtain a query vector and a plurality of reference vectors, wherein the query vector is obtained based on the image to be retrieved, and the reference vector is obtained based on the images included in the gallery; The matrix calculation unit is used to perform matrix multiplication calculation on a left matrix constructed by the vector calculation unit based on the query vector and a right matrix constructed by the vector calculation unit based on the multiple reference vectors, so as to obtain a Hamming distance between the query vector and each of the reference vectors; The vector calculation unit is further used to obtain a retrieval result according to the sorting results of the multiple Hamming distances, and the retrieval result includes images in the gallery that are similar to the image to be retrieved.

10. The device according to claim 9, characterized in that The query vector and the reference vector are both N-dimensional vectors, where N is a positive integer, and the matrix calculation unit is specifically used for: Performing matrix multiplication calculation on the left matrix and the right matrix to obtain a first calculation result; The Hamming distance between the query vector and each of the reference vectors is obtained according to the first calculation result and N.

11. The device according to claim 10, characterized in that The query vector and the reference vector are both binary vectors, and the value of each dimension of the binary vector is 0 or 1. The value of each element in the left matrix is ​​-a or a, where a is a positive integer. The position of the element with a value of -a in the left matrix is ​​obtained based on the position of the element with a value of 0 in the query vector, and the position of the element with a value in the left matrix is ​​obtained based on the position of the element with a value of 1 in the query vector. The value of each element in the right matrix is ​​-a or a, wherein the position of the element with a value of -a in the right matrix is ​​obtained based on the position of the element with a value of 0 in each of the reference vectors, and the position of the element with a value in the right matrix is ​​obtained based on the position of the element with a value of 1 in each of the reference vectors.

12. The device according to claim 11, characterized in that The value of a is 1.

13. The device according to any one of claims 10 to 12, characterized in that The vector calculation unit is specifically used for: Copy the sequence numbers corresponding to the multiple Hamming distances to a target matrix of N+1 rows and K columns, where K is a positive integer, the values ​​of the Hamming distances corresponding to the sequence numbers stored in each row of the target matrix are the same, and the value of the Hamming distance corresponding to any sequence number stored in the previous row of the N+1 rows is less than the value of the Hamming distance corresponding to any sequence number stored in the next row; K serial numbers are obtained according to the target matrix, and the K serial numbers are used to obtain the search results.

14. The device according to claim 13, characterized in that The vector calculation unit is specifically used for: Traversing the multiple Hamming distances to count the number of current target Hamming distances, where the target Hamming distance is any one of the multiple Hamming distances; Obtaining an offset vector of the target Hamming distance according to the current number of the target Hamming distances, the value of the target Hamming distance and K; The serial number corresponding to the target Hamming distance is copied to a specified position of the target matrix, where the specified position is obtained based on the offset vector.

15. The device according to claim 14, characterized in that The vector calculation unit is specifically used for: The scatter instruction is called to copy the serial number corresponding to the target Hamming distance to the specified position of the target matrix.

16. The device according to any one of claims 13 to 15, characterized in that The device also includes a main control logic unit, The main control logic unit is used to divide the multiple reference vectors into multiple groups, each of the groups includes at least one reference vector, and the multiple groups include a first group and a second group; The vector calculation unit is specifically used to copy the serial number corresponding to the Hamming distance between the query vector and each reference vector in the first group to the target matrix of N+1 rows and K columns when the matrix calculation unit obtains the Hamming distance between the query vector and each reference vector in the second group.

17. A system for image retrieval, characterized in that: The system includes a vector calculation unit and a matrix calculation unit, The vector calculation unit is used to obtain a query vector and a plurality of reference vectors, wherein the query vector is obtained based on the image to be retrieved, and the reference vector is obtained based on the images included in the gallery; The matrix calculation unit is used to perform matrix multiplication calculation on a left matrix constructed by the vector calculation unit based on the query vector and a right matrix constructed by the vector calculation unit based on the multiple reference vectors, so as to obtain a Hamming distance between the query vector and each of the reference vectors; The vector calculation unit is further used to obtain a retrieval result according to the sorting results of the multiple Hamming distances, and the retrieval result includes images in the gallery that are similar to the image to be retrieved.

18. A computer-readable storage medium, characterized in that: The invention comprises a program, which, when being executed on a computer, causes the computer to execute the method according to any one of claims 1 to 8.

Images