Vector retrieval acceleration system and method based on integration of storage and calculation

Abstract

The invention discloses a vector retrieval acceleration system and method based on storage and calculation integration, and the system comprises a server which is used for receiving a target vector, selecting a target control chip from at least one control chip in communication connection with the server, and transmitting the target vector to the target control chip; the target control chip is used for receiving a target vector and sending the target vector to the at least one storage and calculation integrated chip; at least one storage and calculation integrated chip which is in communication connection with the target control chip and is used for receiving the target vector, performing similarity calculation on the target vector and the plurality of bottom library vectors, obtaining M bottom library vectors having the highest similarity with the target vector, and sending the M bottom library vectors having the highest similarity with the target vector to the target control chip; and the target control chip is used for recalculating the similarity between each bottom library vector in the M bottom library vectors and the target vector, obtaining K bottom library vectors with the highest similarity with the target vector, and sending the K bottom library vectors as final retrieval results to the server.

Description

Technical Field

[0001] This application relates to the field of vector retrieval technology, and more specifically, to a vector retrieval acceleration system and method based on in-memory computing. Background Technology

[0002] In the traditional von Neumann computing architecture, memory and processor are separate, connected via a data bus, requiring data to be transferred back and forth between memory and processor for processing. However, with the rise of big data applications, the transmission and processing of massive amounts of data presents the traditional von Neumann computing architecture with dual challenges of bandwidth and power consumption, known as the memory wall and power wall problems, respectively. To address these challenges, in recent years, inspired by the ability of synapses in the human brain to simultaneously perform memory and computation, Processing-In-Memory (PIM) technology in computer architecture has received widespread research attention. PIM chips integrate storage and computing units, thus reducing latency and energy consumption caused by data transfer between storage and computing units.

[0003] Although in-memory computing chips can speed up vector retrieval, their limited computational accuracy means they can only meet relatively low-precision retrieval needs, not high-precision ones. Summary of the Invention

[0004] This application provides a vector retrieval acceleration system and method based on in-memory computing, which can further improve retrieval accuracy while meeting retrieval efficiency requirements.

[0005] The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a vector retrieval acceleration system based on in-memory computing, the system comprising: at least one server, at least one control chip and at least one in-memory computing chip, each server being communicatively connected to at least one control chip, and each control chip being communicatively connected to at least one in-memory computing chip;

[0007] The server is configured to receive a target vector, select a target control chip from at least one control chip that is communicatively connected to itself, and send the target vector to the target control chip.

[0008] The target control chip is used to receive the target vector sent by the server and send the target vector to at least one of the in-memory computing chips that are communicatively connected to itself.

[0009] At least one in-memory computing chip that is communicatively connected to the target control chip is used to receive the target vector, perform similarity calculations on the target vector and multiple stored base vectors respectively, obtain the M base vectors with the highest similarity to the target vector, and send the M base vectors with the highest similarity to the target vector to the target control chip.

[0010] The target control chip is further configured to receive the M base library vectors with the highest similarity to the target vector, recalculate the similarity between each of the M base library vectors and the target vector, obtain the K base library vectors with the highest similarity to the target vector, and send the K base library vectors with the highest similarity to the target vector as the final retrieval result to the server communicatively connected to the target control chip, wherein M is greater than K, and both M and K are positive integers.

[0011] In one possible implementation, all control chips are divided into N levels, where N is greater than or equal to 2. Each server is communicatively connected to at least one control chip in the first level, and starting from the first level, each control chip in the current level is communicatively connected to at least one control chip in the adjacent next level, until each control chip in the Nth level is communicatively connected to at least one of the in-memory computing chips.

[0012] The server is configured to select a target control chip at the first level from at least one control chip at the first level, and send the target vector to the target control chip at the first level.

[0013] The first-level target control chip is used to select a second-level target control chip from at least one second-level control chip, and send the target vector to the second-level target control chip, until the target vector is sent to the Nth-level target control chip;

[0014] The target control chip at the Nth level is used to send the target vector to at least one of the in-memory computing chips that are communicatively connected to itself.

[0015] At least one in-memory computing chip that is communicatively connected to the target control chip at the Nth level is used to send the M base library vectors with the highest similarity to the target vector to the target control chip at the Nth level after obtaining the M base library vectors with the highest similarity to the target vector.

[0016] The target control chip at the Nth level is also used to recalculate the similarity between each of the M base library vectors and the target vector, obtain the K base library vectors with the highest similarity to the target vector, and report the K base library vectors with the highest similarity to the target vector as the final retrieval result to the target control chip at the N-1th level, and report them layer by layer until they reach the server.

[0017] In one possible implementation, the server communicates with the first-level control chip via a PCIe interface;

[0018] The control chips at each level communicate with each other through a custom interface, and / or the control chip at the Nth level communicates with the in-memory computing chip through a custom interface. The communication protocol of the custom interface can merge the data stream and the control stream into a communication bus, and communication is achieved by encoding and decoding data packets including the data stream and the control stream.

[0019] In one possible implementation, when there are multiple in-memory computing chips that are communicatively connected to the target control chip, the multiple in-memory computing chips that are communicatively connected to the target control chip realize distributed storage base vector and distributed computing.

[0020] In one possible implementation, a target in-memory computing chip among a plurality of in-memory computing chips communicatively connected to the target control chip is configured to acquire the distributed computing results of other in-memory computing chips communicatively connected to the target control chip and its own distributed computing results, and to acquire the M base library vectors with the highest similarity to the target vector from all distributed computing results, and send them to the target control chip. Each distributed computing result includes the similarity between all its stored base library vectors and the target vector, or the similarity between all its stored base library vectors and the target vector that is greater than or equal to a similarity threshold.

[0021] Secondly, embodiments of this application provide a vector retrieval acceleration method based on in-memory computing. The method is applied to the target control chip, which is selected from at least one control chip communicatively connected to the server after the server receives the target vector. The method includes:

[0022] The target control chip receives the target vector sent by the server and sends the target vector to at least one in-memory computing chip that is connected to it. After receiving the target vector, the at least one in-memory computing chip that is connected to it performs similarity calculation on the target vector and multiple stored base vectors respectively to obtain the M base vectors with the highest similarity to the target vector.

[0023] The target control chip receives the M base database vectors sent by at least one in-memory computing chip that is communicatively connected to itself;

[0024] The target control chip recalculates the similarity between each of the M base vectors and the target vector to obtain the K base vectors with the highest similarity to the target vector;

[0025] The target control chip sends the K base library vectors with the highest similarity to the target vector as the final retrieval results to the server that is communicatively connected to the target control chip, wherein M is greater than K, and both M and K are positive integers.

[0026] In one possible implementation, all control chips are divided into N levels, where N is greater than or equal to 2. Each server is communicatively connected to at least one control chip in the first level, and starting from the first level, each control chip in the current level is communicatively connected to at least one control chip in the adjacent next level, until each control chip in the Nth level is communicatively connected to at least one of the in-memory computing chips.

[0027] The target control chip receives the target vector sent by the server and sends the target vector to at least one in-memory computing chip communicatively connected to itself. Upon receiving the target vector, the at least one in-memory computing chip performs similarity calculations between the target vector and multiple stored base database vectors to obtain the M base database vectors with the highest similarity to the target vector, including:

[0028] The first-level target control chip receives the target vector sent by the server, and selects a second-level target control chip from at least one second-level control chip, and sends the target vector to the second-level target control chip, until the target vector is sent to the Nth-level target control chip. The Nth-level target control chip sends the target vector to at least one in-memory computing chip that is communicatively connected to itself, so that after receiving the target vector, the at least one in-memory computing chip that is communicatively connected to itself performs similarity calculation on the target vector and multiple stored base database vectors respectively, and obtains the M base database vectors with the highest similarity to the target vector.

[0029] The target control chip receives the M base library vectors sent by at least one in-memory computing chip that is communicatively connected to it, including:

[0030] The target control chip at the Nth level receives the M base library vectors sent by at least one in-memory computing chip that is connected to it.

[0031] In one possible implementation, the server communicates with the first-level control chip via a PCIe interface;

[0032] The control chips at each level communicate with each other through a custom interface, and / or the control chip at the Nth level communicates with the in-memory computing chip through a custom interface. The communication protocol of the custom interface can merge the data stream and the control stream into a communication bus, and communication is achieved by encoding and decoding data packets including the data stream and the control stream.

[0033] In one possible implementation, when there are multiple in-memory computing chips that are communicatively connected to the target control chip, the multiple in-memory computing chips that are communicatively connected to the target control chip realize distributed storage base vector and distributed computing.

[0034] In one possible implementation, the M base library vectors are the M base library vectors with the highest similarity to the target vector obtained by the target in-memory computing chip from all the distributed computing results of the other in-memory computing chips connected to the target control chip and its own distributed computing results. Each distributed computing result includes the similarity between all the base library vectors stored in itself and the target vector, or the similarity between all the base library vectors stored in itself and the target vector that is greater than or equal to a similarity threshold.

[0035] The in-memory computing-based vector retrieval acceleration system and method provided in this application, after the server receives the target vector, can first forward the target vector to at least one in-memory computing chip through a target control chip selected from at least one control chip connected to the server. Then, the at least one in-memory computing chip calculates the similarity between the target vector and multiple base database vectors, and selects the M base database vectors with the highest similarity to the target vector. Finally, the target control chip recalculates the similarity between the M base database vectors and the target vector, and selects the K base database vectors with the highest similarity to the target vector from the M base database vectors and feeds them back to the server. This realizes the function of first performing coarse retrieval with a high-efficiency in-memory computing chip, and then performing fine retrieval with a high-precision control chip, thereby further improving retrieval accuracy while meeting retrieval efficiency requirements.

[0036] Furthermore, the technical effects that can be achieved by the embodiments of this application include:

[0037] 1. To enable rapid parallel processing of more retrieval tasks, a larger number of control chips can be deployed. To allow the server to quickly select a suitable target control chip from among these chips, all control chips can be divided into N levels, where N is greater than or equal to 2. Each server communicates with at least one control chip in the first level. Starting from the first level, each control chip in the current level communicates with at least one control chip in the next adjacent level, until each control chip in the Nth level communicates with at least one in-memory computing chip. This reduces both the selection range of the server and the selection range of the target control chip in each level, thereby improving the selection efficiency of the target control chip that communicates directly with the in-memory computing chip.

[0038] 2. Compared with interfaces that separate data flow and control flow for communication, the advantages of using a custom interface that integrates data flow and control flow for communication include: (1) Using a simpler communication protocol to achieve more functions; (2) Effectively reducing the number of pins used and reducing the area: Because the accelerator card in this system has many interconnected devices to improve the search level and speed, the number of pins required for the devices is relatively high. In order to maximize the number of devices and improve the computing power, it is necessary to minimize the number of pins used. (3) Parallel transmission can be achieved through this method, which has lower requirements for the chip. The chip does not need to use a high-speed interface, thereby effectively reducing the chip cost.

[0039] 3. By distributing all base vectors to multiple in-memory computing chips and performing distributed computing, the efficiency of obtaining the M base vectors with the highest similarity to the target vector can be improved. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0041] Figure 1 A schematic diagram of the architecture of a vector retrieval acceleration system based on in-memory computing provided for embodiments of this application;

[0042] Figure 2 A schematic diagram of another in-memory computing-based vector retrieval acceleration system provided for embodiments of this application;

[0043] Figure 3 A system architecture example diagram including a grid-based accelerator card is provided for embodiments of this application;

[0044] Figure 4 Another system architecture example diagram including a grid-based accelerator card is provided for embodiments of this application;

[0045] Figure 5 A system architecture example diagram including a hierarchical accelerator card is provided for embodiments of this application;

[0046] Figure 6 This is a flowchart illustrating a method for accelerating vector retrieval based on in-memory computing, provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0048] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0049] This application provides a vector retrieval acceleration system based on in-memory computing, such as... Figure 1 As shown, the system includes: at least one server 110 (one server is used as an example in the figure), at least one control chip 120 and at least one in-memory computing chip 130. Each server 110 is communicatively connected to at least one control chip 120, and each control chip 120 is communicatively connected to at least one in-memory computing chip 130.

[0050] Server 110 is used to receive a target vector and select a target control chip 120 from at least one control chip 120 that is communicatively connected to itself, and send the target vector to the target control chip 120.

[0051] The target control chip 120 is used to receive the target vector sent by the server 110 and send the target vector to at least one in-memory computing chip 130 that is connected to itself in communication.

[0052] At least one in-memory computing chip 130, which is communicatively connected to the target control chip 120, is used to receive the target vector, perform similarity calculations on the target vector and multiple stored base library vectors respectively, obtain the M base library vectors with the highest similarity to the target vector, and send the M base library vectors with the highest similarity to the target vector to the target control chip 120.

[0053] The target control chip 120 is also used to receive the M base library vectors with the highest similarity to the target vector, and recalculate the similarity between each base library vector and the target vector to obtain the K base library vectors with the highest similarity to the target vector. The K base library vectors with the highest similarity to the target vector are sent as the final retrieval results to the server 110 which is connected to the target control chip 120, where M is greater than K and both M and K are positive integers.

[0054] The embodiments of this application can be applied to various vector retrieval scenarios, such as cross-modal recognition, face and body recognition, fuzzy text recognition, and other scenarios where target retrieval is completed through vectors. For example, when the embodiments of this application are applied to a face and body recognition scenario, the target vector can be a feature vector extracted from a target face (or body) image, and the base vector can be a feature vector extracted from face (or body) images stored in the base database. When the embodiments of this application are applied to a fuzzy text recognition scenario, the target vector can be a feature vector extracted from the target text, and the base vector can be a feature vector extracted from text stored in the base database.

[0055] In this embodiment, the control chip 120 includes an FPGA (Field Programmable Gate Array) chip, an MCU (Micro Control Unit) chip, a PLD (Programmable Logic Device) chip, an ASIC (Application Specific Integrated Circuit) chip, etc. The in-memory computing chip 130 includes FeRAM (Ferroelectric Random Access Memory), RRAM (Resistive Random Access Memory), FLASH (Flash Memory), etc. Both the control chip 120 and the in-memory computing chip 130 can be installed on an accelerator card. The control chip 120 has higher computational accuracy than the in-memory computing chip 130, but its computational efficiency is lower.

[0056] Although the in-memory computing chip 130 has higher computing efficiency than other control chips 120, due to the inherent physical characteristics of the in-memory computing chip 130, there will be certain errors in the ADC writing and reading of data, and the data format used is int8 type. On the other hand, the other control chips 120 have data format FP16 data type, so the other control chips 120 can retain more decimal places, are not affected by too much data distribution, and have no read and write errors. Therefore, the computing accuracy of the in-memory computing chip 130 is actually not as good as that of the other control chips 120.

[0057] To balance computational efficiency and retrieval accuracy, this embodiment combines the in-memory computing chip 130 and the control chip 120. It provides a method where the in-memory computing chip 130 performs a coarse-grained retrieval to obtain the M base vectors with the highest similarity to the target vector, and then the control chip 120 performs a fine-grained retrieval to extract the K base vectors with the highest similarity to the target vector from these M base vectors as the final retrieval result.

[0058] Here, K represents the number of search results requested by the user, and M represents the number determined based on historical experience that can meet the requirements of search accuracy and search efficiency. The value of M may vary when the total number of base database vectors is different.

[0059] Furthermore, when there are multiple control chips 120 that are connected to the server 110, the server 110 can select a target control chip 120 from the multiple control chips 120. The target control chip 120 can be one control chip 120 or multiple control chips 120. Moreover, the selection strategy of the server 110 is not limited in this application embodiment, including but not limited to: random selection, selection based on at least one dimension such as the idle state of the control chip 120 and memory usage rate.

[0060] The in-memory computing-based vector retrieval acceleration system provided in this application embodiment, after the server 110 receives the target vector, can first forward the target vector to at least one in-memory computing chip 130 through a target control chip selected from at least one control chip 120 that is communicatively connected to the server 110. Then, the at least one in-memory computing chip 130 calculates the similarity between the target vector and multiple base library vectors respectively, and selects the M base library vectors with the highest similarity to the target vector. Finally, the target control chip 120 recalculates the similarity between the M base library vectors and the target vector, and selects the K base library vectors with the highest similarity to the target vector from the M base library vectors and feeds them back to the server 110. This realizes the function of first performing a coarse search with the high computational efficiency of the in-memory computing chip 130, and then performing a fine search with the high computational accuracy of the control chip 120, thereby further improving the search accuracy while meeting the search efficiency requirements.

[0061] In one possible implementation, to enable rapid parallel processing of more retrieval tasks, a larger number of control chips can be deployed. Furthermore, to allow the server to quickly select a suitable target control chip from among these chips, such as... Figure 2 As shown, all control chips 120 can be divided into N levels (3 levels are used as an example in the figure), where N is greater than or equal to 2. Each server 110 is connected to at least one control chip 120 in the first level. Starting from the first level, each control chip 120 in the current level is connected to at least one control chip 120 in the next adjacent level, until each control chip 120 in the Nth level is connected to at least one in-memory computing chip 130.

[0062] Server 110 is used to select a target control chip 120 of the first level from at least one control chip 120 of the first level and send the target vector to the target control chip 120 of the first level.

[0063] The first-level target control chip 120 is used to select the second-level target control chip 120 from at least one second-level control chip 120, and send the target vector to the second-level target control chip 120 until the target vector is sent to the Nth-level target control chip 120.

[0064] The target control chip 120 at the Nth level is used to send the target vector to at least one in-memory computing chip 130 that is connected to itself in communication.

[0065] At least one in-memory computing chip 130, which is communicatively connected to the target control chip 120 at the Nth level, is used to send the M base library vectors with the highest similarity to the target vector to the target control chip 120 after obtaining the M base library vectors with the highest similarity to the target vector.

[0066] The target control chip 120 at the Nth level is also used to recalculate the similarity between each of the M base library vectors and the target vector, obtain the K base library vectors with the highest similarity to the target vector, and report the K base library vectors with the highest similarity to the target vector as the final retrieval result to the target control chip 120 at the N-1th level, and report them layer by layer until they reach the server 110.

[0067] In this embodiment, the total number of control chips 120 at each level can be the same or different. The value of N can be determined based on practical experience and actual needs. The selection strategy for each level's target control chip 120 when selecting the next level's target control chip 120 can be the same as or different from the selection strategy of the server 110 when selecting the first level's target control chip 120. For example, the selection strategy between levels may include, but is not limited to: random selection, or selection based on at least one dimension such as the idle state of the control chip 120 or memory occupancy rate.

[0068] In one possible implementation, server 110 communicates with first-level control chip 120 via a PCIE (Peripheral Component Interconnect Express) interface.

[0069] The control chips 120 at each level communicate with each other through a custom interface, and / or the control chip 120 at the Nth level communicates with the in-memory computing chip 130 through a custom interface. The communication protocol of the custom interface can merge the data stream and the control stream into a communication bus, and communication is achieved by encoding and decoding data packets including the data stream and the control stream.

[0070] The data flow can include various instructions (such as programming instructions, calculation instructions, etc.), data information, addresses, etc.; the control flow can include register initialization, debugging control instructions, data transmitted through communication lines between the server and the control chip and between the in-memory computing chip (including target vectors and control instructions, but without a large amount of data such as weight values ​​or base library vectors, which can effectively reduce the bandwidth of the data transmission lines), etc.

[0071] Compared with interfaces that separate data flow and control flow for communication, the advantages of using the above-mentioned custom interface for communication include: (1) more functions can be implemented using a simpler communication protocol; (2) the number of pins can be effectively reduced, thus reducing the area: because the accelerator card of this system has many interconnected devices to improve the search level and speed, the number of pins required for the devices is relatively high. In order to maximize the number of devices and improve the computing power, the number of pins should be minimized; (3) parallel transmission can be achieved through this method, which has lower requirements for the chip. The chip does not need to use a high-speed interface, thus effectively reducing the chip cost.

[0072] In one possible implementation, in order to save the overall storage space of multiple in-memory computing chips 130 storing a large number of base vectors, when there are multiple in-memory computing chips 130 that are communicatively connected to the target control chip 120, the multiple in-memory computing chips 130 that are communicatively connected to the target control chip 120 realize distributed storage of base vectors and distributed computing.

[0073] In this case, the target in-memory computing chip 130, which is connected to the target control chip 120, can obtain the distributed computing results of the other in-memory computing chips 130 connected to the target control chip 120 and its own distributed computing results, and obtain the M base library vectors with the highest similarity to the target vector from all the distributed computing results, and send them to the target control chip 120.

[0074] The target in-memory computing chip 130 can be a pre-specified in-memory computing chip 130 or a real-time selected in-memory computing chip 130. For example, the target control chip 120 can select the target in-memory computing chip 130 from multiple in-memory computing chips 130, and the selection strategy includes, but is not limited to: random selection, selection based on at least one dimension such as the idle state of the in-memory computing chip 130 and memory occupancy rate.

[0075] Each distributed computing result includes the similarity between each of its own stored base vectors and the target vector, or the similarity between the target vector and all stored base vectors that is greater than or equal to a similarity threshold. The similarity threshold can be a default value set based on experience. However, if the total number of similarities greater than or equal to the similarity threshold across all in-memory computing chips 130 is less than M, the similarity threshold can be lowered, and the total number of similarities greater than or equal to the updated similarity threshold can be counted again until the total number is greater than or equal to M.

[0076] It should be added that the distributed architecture in this application includes various structural methods such as parallel, grid, hierarchical, and partitioned architectures, and the control chip and in-memory computing chip can be deployed on the accelerator card. Furthermore, multiple control chips can also be arranged in a distributed manner; this application does not impose any restrictions on this.

[0077] For example, Figure 3 and Figure 4 These are grid-style accelerator card structures. Figure 5 It is a hierarchical accelerator card structure.

[0078] The in-memory computing chip inside the accelerator card includes at least the following features:

[0079] (1) The chips are arranged in a distributed manner, with adjacent chips connected together.

[0080] (2) The in-memory computing chip has at least one (two) communication interface, which is connected to the control chip and / or to the distributed adjacent in-memory computing chips.

[0081] (3) The in-memory computing chip structure includes at least: a computing module, a storage module, and an interface module;

[0082] The storage module is used to store base vectors, weight data, etc.

[0083] The computing module is used for data calculation, which can be implemented through analog computing and / or digital computing. Analog computing can be achieved through memristor arrays, while digital computing can be achieved through accumulator circuit structures.

[0084] (4) In-memory computing chip refers to any chip that can simultaneously realize storage and computing functions. The above computing modules, storage modules and interface modules can be packaged on the same chip or tape through monolithic integration, 3D packaging and other forms, including but not limited to: memristor in-memory computing chip, near-memory chip, ferroelectric in-memory computing chip, etc.

[0085] (5) Data transmitted between the server and the accelerator card, and between the in-memory computing chip, including: target vector and control instructions (not containing a large amount of data such as weight values ​​or base data, which can effectively reduce the bandwidth of the data transmission line).

[0086] In one possible implementation, data quantization is often required when performing vector retrieval on an in-memory computing chip to meet its own accuracy requirements. However, in traditional quantization methods, not only is the input data quantized, but the quantization parameters required for quantization are also determined based on empirical values. Therefore, using this quantization method leads to a decrease in the accuracy of vector retrieval. To improve the computational accuracy of the in-memory computing chip, and thus further improve the overall computational accuracy of vector retrieval, this application embodiment can pre-train a set of quantization parameters and perform quantization and retrieval based on this set of quantization parameters. The training method of the quantization parameters and the vector retrieval process of the in-memory computing chip are described below.

[0087] (I) Training methods for quantization parameters

[0088] (A1) Based on the initial input scaling parameter and the initial input translation parameter, the first target vector is linearly transformed to obtain the target vector after linear transformation as the second target vector, and the second target vector is quantized based on the input quantization function to obtain the quantized target vector as the third target vector.

[0089] Here, the first target vector is the original target vector. The initial input scaling parameter is used to scale the first target vector, representing the scaling amount. The initial input translation parameter is used to translate the input vector, representing the translation amount. The input quantization function is used to quantize the second target vector.

[0090] This application embodiment allows for arbitrary direction and magnitude translation of the first target vector, as long as the translation direction and magnitude of the first target image and the first base vector remain consistent. Furthermore, when performing a linear transformation on the first target vector, translation followed by scaling, or scaling followed by translation, can be performed first.

[0091] (A2) Based on the initial input translation parameters and the initial weight scaling parameters, the first base vector is linearly transformed to obtain the linearly transformed base vector as the second base vector, and the second base vector is quantized based on the weight quantization function to obtain the quantized base vector as the third base vector.

[0092] Here, the first base vector is the original base vector. The initial weight scaling parameter is used to scale the input first base vector, and the weight quantization function is used to quantize the input second base vector.

[0093] When performing a linear transformation on the first base vector, you can either translate first and then scale, or scale first and then translate, as long as the order of operations is consistent with that of the linear transformation on the first target vector.

[0094] (A3) Calculate the first equivalent Euclidean distance based on the initial weight scaling parameter, the initial input scaling parameter, the third target vector, and the third base vector. Perform a linear transformation on the first equivalent Euclidean distance based on the initial output scaling parameter and the initial output translation parameter to obtain the first equivalent Euclidean distance after linear transformation as the second equivalent Euclidean distance. Quantize the first equivalent Euclidean distance after linear transformation based on the output quantization function to obtain the quantized equivalent Euclidean distance as the third equivalent Euclidean distance.

[0095] The first equivalent Euclidean distance is the distance between the third target vector and the third base vector, adjusted according to the initial weight scaling parameters and the initial input scaling parameters. The equivalent Euclidean distance between the two vectors is derived from the Euclidean distance formula and can replace the Euclidean distance in evaluating the similarity between the two vectors. The initial output scaling parameters are used to scale the output vector, the initial output translation parameters are used to translate the output vector, and the output quantization function is used to quantize the output vector.

[0096] The first equivalent Euclidean distance formula includes:

[0097]

[0098] Where d1 represents the first equivalent Euclidean distance, Represents the third target vector. Denotes the third base vector, k w k represents the initial weight scaling parameter.i This represents the initial input scaling parameters.

[0099] (A4) Based on the first target vector, initial input translation parameters, initial input scaling parameters, initial weight scaling parameters, initial output scaling parameters, and initial output translation parameters, the third equivalent Euclidean distance is restored to obtain the quantized Euclidean distance between the first target vector and the first base vector.

[0100] Formulas for restoring the third equivalent Euclidean distance can include:

[0101]

[0102] Among them, y q This represents the quantized Euclidean distance. Let λ represent the first target vector. i k represents the initial input translation parameter. i k represents the initial input scaling parameter. w k represents the initial weight scaling parameter. o This represents the initial output scaling parameters, y3 represents the third equivalent Euclidean distance, and λ represents the initial output scaling parameters. o This represents the initial output translation parameters.

[0103] (A5) Calculate the current quantization loss based on the original Euclidean distance and the quantized Euclidean distance, and use the optimization method and the current quantization loss to adjust the initial input scaling parameter, initial input translation parameter, initial weight scaling parameter, initial output scaling parameter and initial output translation parameter until the current quantization loss is less than or equal to the preset loss threshold, and obtain the target input scaling parameter, target input translation parameter, target weight scaling parameter, target output scaling parameter and target output translation parameter.

[0104] Wherein, the original Euclidean distance is the true value of the Euclidean distance before quantization, that is, the value of the original Euclidean distance is the same as... The values ​​are the same.

[0105] The current formula for calculating quantization loss may include:

[0106] l(k i ,λ i ,λ w ,k o ,λ o ) = abs(y q -y)

[0107] Where, l(k) i ,λ i ,λ w ,k o ,λ o) represents the quantization loss function, y q y represents the quantized Euclidean distance, and y represents the original Euclidean distance.

[0108] Optimization methods include non-gradient optimization methods, such as CMA (Covariance Matrix Adaptation) and grid search methods. Specifically, CMA can be CMA-ES (Covariance Matrix Adaptation Evolution strategies). The preset loss threshold can be an empirical value.

[0109] (ii) The specific calculation process by which at least one in-memory computing chip obtains the M base vectors with the highest similarity to the target vector by calculating the similarity between the target vector and multiple stored base vectors includes:

[0110] (B1) Based on the target input scaling parameter and the target input translation parameter, the original target vector is linearly transformed to obtain the linearly transformed target vector, and the linearly transformed target vector is quantized based on the input quantization function to obtain the quantized target vector.

[0111] The target input scaling parameter, target input translation parameter, target weight scaling parameter, target output scaling parameter, and target output translation parameter are determined by the quantization parameter training method provided in any of the above embodiments.

[0112] This step uses the same specific calculation formula as step (A1) above, the only difference being that step (A1) uses the initial input scaling parameter and the initial input translation parameter, while this step uses the target input scaling parameter and the target input translation parameter.

[0113] (B2) Based on the target input translation parameters and target weight scaling parameters, the original base library vector is linearly transformed to obtain the linearly transformed base library vector, and the linearly transformed base library vector is quantized based on the weight quantization function to obtain the quantized base library vector.

[0114] This step uses the same specific calculation formula as step (A2) above, the only difference being that step (A2) uses the initial input translation parameters and the initial weight scaling parameters, while this step uses the target input translation parameters and the target weight scaling parameters.

[0115] (B3) Calculate the fourth equivalent Euclidean distance based on the target weight scaling parameter, the target input scaling parameter, the quantized target vector and the quantized base vector. Perform a linear transformation on the fourth equivalent Euclidean distance based on the target output scaling parameter and the target output translation parameter. Quantize the first equivalent Euclidean distance after the linear transformation based on the output quantization function to obtain the quantized equivalent Euclidean distance.

[0116] The method for calculating the fourth equivalent Euclidean distance based on the target weight scaling parameter, the target input scaling parameter, the quantized target vector, and the quantized base vector is the same as the method for calculating the first equivalent Euclidean distance based on the initial weight scaling parameter, the initial input scaling parameter, the third target vector, and the third base vector, and will not be described again here.

[0117] (B4) By comparing the quantized equivalent Euclidean distances corresponding to different original base vectors, the M original base vectors with the highest similarity to the original target vector are determined.

[0118] After calculating the quantized equivalent Euclidean distances corresponding to each original base vector, the quantized equivalent Euclidean distances corresponding to each original base vector can be sorted, and the M quantized equivalent Euclidean distances with the largest values ​​can be found. The original base vectors corresponding to these M quantized equivalent Euclidean distances are then determined as the M original base vectors with the highest similarity to the original target vector.

[0119] The quantization parameters used in this application include not only input-related quantization parameters but also output-related quantization parameters. These quantization parameters are not empirical values ​​but are automatically adjusted using optimization methods. Therefore, when the vectors in the vector retrieval process are quantized according to the target input scaling parameters, target input translation parameters, target weight scaling parameters, target output scaling parameters, and target output translation parameters, the final quantized equivalent Euclidean distance can be made closer to the original equivalent Euclidean distance, or the final quantized Euclidean distance can be made closer to the original Euclidean distance. Thus, whether the equivalent formula of the original Euclidean distance formula is used for similarity comparison or the original Euclidean distance formula is used directly for similarity comparison, the accuracy of vector retrieval by the in-memory computing chip can be improved.

[0120] Another embodiment of this application also provides a vector retrieval acceleration method based on in-memory computing. This method is applied to the target control chip, which is selected from at least one control chip communicatively connected to the server after the server receives the target vector. Figure 6 As shown, the method may include:

[0121] S210: The target control chip receives the target vector sent by the server and sends the target vector to at least one in-memory computing chip that is connected to it. After receiving the target vector, the at least one in-memory computing chip that is connected to it performs similarity calculations on the target vector and multiple stored base library vectors respectively, and obtains the M base library vectors with the highest similarity to the target vector.

[0122] Control chips include FPGA chips, MCU chips, PLD chips, ASIC chips, etc. In-memory computing chips include memristor in-memory computing chips, near-memory computing chips, and in-memory processing chips, etc. Both control chips and in-memory computing chips can be installed on accelerator cards.

[0123] When there are multiple control chips communicating with the server, the server can select a target control chip from the multiple control chips. The target control chip can be one control chip or multiple control chips. Furthermore, the selection strategy of the server is not limited in this application embodiment, including but not limited to: random selection, selection based on at least one dimension such as the idle state of the control chip, memory usage rate, etc.

[0124] When multiple in-memory computing chips are connected to the target control chip, the multiple in-memory computing chips connected to the target control chip realize distributed storage base vector and distributed computing.

[0125] The M base library vectors are selected by the target in-memory computing chip from among multiple in-memory computing chips that are communicatively connected to the target control chip. The target chip obtains the distributed computing results from the other in-memory computing chips and its own distributed computing results, and then selects the M base library vectors with the highest similarity to the target vector from all the distributed computing results. Each distributed computing result includes the similarity between all its stored base library vectors and the target vector, or the similarity between its stored base library vectors and the target vector that is greater than or equal to a similarity threshold. The target in-memory computing chip can be a pre-specified chip or a chip selected in real-time. For example, the target control chip can select the target in-memory computing chip from multiple chips, and this selection strategy includes, but is not limited to, random selection, or selection based on at least one dimension such as the chip's idle state or memory usage.

[0126] To enable rapid parallel processing of more retrieval tasks, a larger number of control chips can be deployed. To allow the server to quickly select a suitable target control chip from among these chips, all control chips can be divided into N levels, where N is greater than or equal to 2. Each server communicates with at least one control chip in the first level. Starting from the first level, each control chip in the current level communicates with at least one control chip in the next adjacent level, until each control chip in the Nth level communicates with at least one in-memory computing chip. This reduces both the selection range of the server and the selection range of the target control chip in each level, thereby improving the selection efficiency of the target control chip that communicates directly with the in-memory computing chip.

[0127] In this scenario of multi-level control chips, the specific implementation method of step S210 includes: the first-level target control chip receives the target vector sent by the server, and selects a second-level target control chip from at least one second-level control chip, and sends the target vector to the second-level target control chip, until the target vector is sent to the Nth-level target control chip. The Nth-level target control chip sends the target vector to at least one in-memory computing chip that is connected to it, so that after receiving the target vector, the at least one in-memory computing chip that is connected to it can perform similarity calculations on the target vector and multiple stored base library vectors respectively, and obtain the M base library vectors with the highest similarity to the target vector.

[0128] The server communicates with the first-level control chip via a PCIe interface. Control chips at different levels can communicate with each other through custom interfaces. These custom interfaces use a communication protocol that merges data and control flows into a single communication bus, achieving communication through encoding and decoding data packets containing both data and control flows. The data flow can include various instructions (such as programming and calculation instructions), data information, and addresses. The control flow can include register initialization, debugging control instructions, and data transmitted via communication lines between the server and the control chip, and between the in-memory computing chip (including target vectors and control instructions, but excluding large amounts of data such as weight values ​​or base vectors, effectively reducing the bandwidth of the data transmission lines).

[0129] Compared to communication protocols that transmit data and control streams separately, this custom interface's communication protocol can effectively reduce the number of pins used, reduce chip area, and achieve parallel transmission. It also has lower requirements for the chip, eliminating the need for a high-speed interface and thus effectively reducing chip cost.

[0130] In this embodiment, the total number of control chips at each level can be the same or different. The value of N can be determined based on practical experience and actual needs. The selection strategy for each level's target control chip when selecting the next level's target control chip can be the same as or different from the server's selection strategy for the first level's target control chip. For example, the selection strategy between levels may include, but is not limited to: random selection, or selection based on at least one dimension such as the control chip's idle state or memory usage.

[0131] S220: The target control chip receives M base library vectors sent by at least one in-memory computing chip that is connected to it.

[0132] When all control chips are divided into N levels, this step specifically involves the target control chip at level N receiving M base library vectors sent by at least one in-memory computing chip that is connected to it.

[0133] To effectively reduce the number of pins used and lower the chip area, the Nth-level control chip and the in-memory computing chip can also communicate through the aforementioned custom interface.

[0134] S230: The target control chip recalculates the similarity between each of the M base vectors and the target vector, and obtains the K base vectors with the highest similarity to the target vector.

[0135] S240: The target control chip sends the K base vectors with the highest similarity to the target vector as the final retrieval results to the server that is connected to the target control chip.

[0136] Where M is greater than K, and both M and K are positive integers.

[0137] When all control chips are divided into N levels, the target control chip at level N can recalculate the similarity between each of the M base vectors and the target vector, obtain the K base vectors with the highest similarity to the target vector, and report the K base vectors with the highest similarity to the target vector as the final retrieval results to the target control chip at level N-1, and report them layer by layer until they reach the server.

[0138] The in-memory computing-based vector retrieval acceleration method provided in this application, after the server receives the target vector, can first forward the target vector to at least one in-memory computing chip through a target control chip selected from at least one control chip connected to the server. Then, the at least one in-memory computing chip calculates the similarity between the target vector and multiple base database vectors, and selects the M base database vectors with the highest similarity to the target vector. Finally, the target control chip recalculates the similarity between the M base database vectors and the target vector, and selects the K base database vectors with the highest similarity to the target vector from the M base database vectors and feeds them back to the server. This realizes the function of first performing a coarse search with a high-efficiency in-memory computing chip, and then performing a fine search with a high-precision control chip, thereby further improving the search accuracy while meeting the search efficiency requirements.

[0139] The above system embodiments correspond to the method embodiments and have the same technical effects. For detailed descriptions, please refer to the system embodiments. Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0140] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A vector retrieval acceleration system based on in-memory computing, characterized in that, The system includes: at least one server, at least one control chip, and at least one in-memory computing chip, wherein each server is communicatively connected to at least one control chip, and each control chip is communicatively connected to at least one in-memory computing chip; The server is configured to receive a target vector, select a target control chip from at least one control chip that is communicatively connected to itself, and send the target vector to the target control chip. The target control chip is used to receive the target vector sent by the server and send the target vector to at least one of the in-memory computing chips that are communicatively connected to itself. At least one in-memory computing chip that is communicatively connected to the target control chip is used to receive the target vector, perform similarity calculations on the target vector and multiple stored base vectors respectively, obtain the M base vectors with the highest similarity to the target vector, and send the M base vectors with the highest similarity to the target vector to the target control chip. The target control chip is further configured to receive the M base library vectors with the highest similarity to the target vector, recalculate the similarity between each of the M base library vectors and the target vector, obtain the K base library vectors with the highest similarity to the target vector, and send the K base library vectors with the highest similarity to the target vector as the final retrieval result to the server communicatively connected to the target control chip, wherein M is greater than K, and both M and K are positive integers.

2. The system according to claim 1, characterized in that, All control chips are divided into N levels, where N is greater than or equal to 2. Each server is communicatively connected to at least one control chip in the first level. Starting from the first level, each control chip in the current level is communicatively connected to at least one control chip in the next adjacent level, until each control chip in the Nth level is communicatively connected to at least one in-memory computing chip. The server is configured to select a target control chip at the first level from at least one control chip at the first level, and send the target vector to the target control chip at the first level. The first-level target control chip is used to select a second-level target control chip from at least one second-level control chip, and send the target vector to the second-level target control chip, until the target vector is sent to the Nth-level target control chip; The target control chip at the Nth level is used to send the target vector to at least one of the in-memory computing chips that are communicatively connected to itself. At least one in-memory computing chip that is communicatively connected to the target control chip at the Nth level is used to send the M base library vectors with the highest similarity to the target vector to the target control chip at the Nth level after obtaining the M base library vectors with the highest similarity to the target vector. The target control chip at the Nth level is also used to recalculate the similarity between each of the M base library vectors and the target vector, obtain the K base library vectors with the highest similarity to the target vector, and report the K base library vectors with the highest similarity to the target vector as the final retrieval result to the target control chip at the N-1th level, and report them layer by layer until they reach the server.

3. The system according to claim 2, characterized in that, The server communicates with the first-level control chip via a PCIe interface; The control chips at each level communicate with each other through a custom interface, and / or the control chip at the Nth level communicates with the in-memory computing chip through a custom interface. The communication protocol of the custom interface can merge the data stream and the control stream into a communication bus, and communication is achieved by encoding and decoding data packets including the data stream and the control stream.

4. The system according to any one of claims 1-3, characterized in that, When there are multiple in-memory computing chips that are communicatively connected to the target control chip, the multiple in-memory computing chips that are communicatively connected to the target control chip realize distributed storage of base library vectors and distributed computing.

5. The system according to claim 4, characterized in that, The target in-memory computing chip, which is communicatively connected to the target control chip, is used to acquire the distributed computing results of other in-memory computing chips that are communicatively connected to the target control chip and its own distributed computing results. It then acquires the M base library vectors with the highest similarity to the target vector from all the distributed computing results and sends them to the target control chip. Each distributed computing result includes the similarity between all the base library vectors stored in itself and the target vector, or the similarity between all the base library vectors stored in itself and the target vector that is greater than or equal to a similarity threshold.

6. A method for accelerating vector retrieval based on in-memory computing, characterized in that, The method is applied to the target control chip, which is selected from at least one control chip that is communicatively connected to the server itself after the server receives the target vector. The method includes: The target control chip receives the target vector sent by the server and sends the target vector to at least one in-memory computing chip that is connected to it. After receiving the target vector, the at least one in-memory computing chip that is connected to it performs similarity calculation on the target vector and multiple stored base vectors respectively to obtain the M base vectors with the highest similarity to the target vector. The target control chip receives the M base database vectors sent by at least one in-memory computing chip that is communicatively connected to itself; The target control chip recalculates the similarity between each of the M base vectors and the target vector to obtain the K base vectors with the highest similarity to the target vector; The target control chip sends the K base library vectors with the highest similarity to the target vector as the final retrieval results to the server that is communicatively connected to the target control chip, wherein M is greater than K, and both M and K are positive integers.

7. The method according to claim 6, characterized in that, All control chips are divided into N levels, where N is greater than or equal to 2. Each server is communicatively connected to at least one control chip in the first level. Starting from the first level, each control chip in the current level is communicatively connected to at least one control chip in the next adjacent level, until each control chip in the Nth level is communicatively connected to at least one in-memory computing chip. The target control chip receives the target vector sent by the server and sends the target vector to at least one in-memory computing chip communicatively connected to itself. Upon receiving the target vector, the at least one in-memory computing chip performs similarity calculations between the target vector and multiple stored base database vectors to obtain the M base database vectors with the highest similarity to the target vector, including: The first-level target control chip receives the target vector sent by the server, and selects a second-level target control chip from at least one second-level control chip, and sends the target vector to the second-level target control chip, until the target vector is sent to the Nth-level target control chip. The Nth-level target control chip sends the target vector to at least one in-memory computing chip that is communicatively connected to itself, so that after receiving the target vector, the at least one in-memory computing chip that is communicatively connected to itself performs similarity calculation on the target vector and multiple stored base database vectors respectively, and obtains the M base database vectors with the highest similarity to the target vector. The target control chip receives the M base library vectors sent by at least one in-memory computing chip that is communicatively connected to it, including: The target control chip at the Nth level receives the M base database vectors sent by at least one in-memory computing chip that is connected to it.

8. The method according to claim 7, characterized in that, The server communicates with the first-level control chip via a PCIe interface; The control chips at each level communicate with each other through a custom interface, and / or the control chip at the Nth level communicates with the in-memory computing chip through a custom interface. The communication protocol of the custom interface can merge the data stream and the control stream into a communication bus, and communication is achieved by encoding and decoding data packets including the data stream and the control stream.

9. The method according to any one of claims 6-8, characterized in that, When there are multiple in-memory computing chips that are communicatively connected to the target control chip, the multiple in-memory computing chips that are communicatively connected to the target control chip realize distributed storage of base library vectors and distributed computing.

10. The method according to claim 9, characterized in that, The M base vectors are the M base vectors with the highest similarity to the target vector obtained by the target in-memory computing chip from the distributed computing results of other in-memory computing chips connected to the target control chip and its own distributed computing results among all distributed computing results. Each distributed computing result includes the similarity between all its stored base vectors and the target vector, or the similarity between all its stored base vectors and the target vector that is greater than or equal to a similarity threshold.

Images