Large-scale data search method, device, memory, readable storage medium and program product for DRAM in-memory computing

Abstract

The present application relates to a large-scale data search method, apparatus, memory, readable storage medium, and program product for DRAM in-memory computing. Applied to the memory, the method includes: upon receiving a query transaction request carrying a query vector from an external device, determining a target computing unit from a plurality of computing units; utilizing the target computing unit to query a plurality of target cluster center vectors from a first mapping table in a plurality of storage units based on the query vector; utilizing the target computing unit to calculate the distance between the query vector and each target cluster center vector to obtain a plurality of distance values; constructing a second mapping table based on the plurality of distance values ​​and the plurality of target cluster center vectors, and storing the second mapping table in a plurality of storage units; utilizing the plurality of storage units and the target computing unit to query based on each basis vector and the second mapping table to obtain R target basis vectors similar to the query vector; and transmitting the R target basis vectors to an external device. This method can reduce search latency.

Description

Technical Field

[0001] The present application relates to the field of search technology, and in particular to a large-scale data search method, device, memory, readable storage medium and program product for DRAM in-memory computing. Background Art

[0002] As a key technology in the field of big data search, the Approximate Nearest Neighbor Search (ANNS) algorithm plays an indispensable role in databases containing massive amounts of data. By quickly and efficiently retrieving objects similar to a given query vector from a high-dimensional vector set, it provides key support in various fields, including information retrieval, recommendation systems, and machine learning.

[0003] In traditional technology, the execution of the ANNS algorithm by a processor (Central Processing Unit, CPU) typically involves calculating and comparing the query vector with the data stored in the vector library. However, when the size of the data set exceeds the capacity of the on-chip cache, the CPU will have to frequently access the memory to read the required data. This frequent memory access not only leads to significant search latency problems, but also increases the energy consumption of the search. Summary of the Invention

[0004] Based on this, it is necessary to provide a large-scale data search method, device, memory, readable storage medium and program product for DRAM in-memory computing that can reduce search delay and energy consumption to address the above technical problems.

[0005] In a first aspect, the present application provides a large-scale data search method for DRAM in-memory computing, which is applied to a memory, wherein the memory includes a dynamic random access memory (DRAM), and the DRAM includes multiple storage units and multiple computing units. Each storage unit stores a first mapping table and multiple quantized basis vectors, and the first mapping table includes multiple one-to-one corresponding subvectors and a first cluster center vector. The method includes:

[0006] After receiving a query transaction request carrying a query vector sent by an external device, a target computing unit is determined from multiple computing units; using the target computing unit, multiple target cluster center vectors are queried from a first mapping table in multiple storage units based on the query vector; using the target computing unit, the distance between the query vector and each target cluster center vector is calculated to obtain multiple distance values; and a second mapping table is constructed based on the multiple distance values ​​and the multiple target cluster center vectors, and the second mapping table is stored in multiple storage units; using the multiple storage units and the target computing unit, based on each basis vector and the second mapping table, R target basis vectors similar to the query vector are queried and obtained; and the R target basis vectors are sent to the external device, where R is a positive integer.

[0007] In one embodiment, a target computing unit is used to query multiple target cluster center vectors from a first mapping table in multiple storage units based on a query vector, including: using the target computing unit, segmenting the query vector to obtain multiple segment vectors; for each segment vector, querying the first mapping table in the multiple storage units for a subvector identical to the segment vector to obtain a target cluster center vector corresponding to the segment vector.

[0008] In one embodiment, the basis vector includes multiple second cluster center vectors; using multiple storage units and target computing units, based on each basis vector and the second mapping table, query to obtain R target basis vectors similar to the query vector, including: for each basis vector, based on the identifier of each second cluster center vector in the basis vector, determine the DRAM row address corresponding to the identifier of each second cluster center vector, based on each DRAM row address, the corresponding storage unit sends the stored distance value to the target computing unit; using the target computing unit, through the Top-R queue, according to the sum of the distance values ​​corresponding to each second cluster center vector in each basis vector, obtain R target basis vectors similar to the query vector.

[0009] In one embodiment, the memory further includes a memory controller; after receiving a query transaction request carrying a query vector sent by an external device, determining a target computing unit from a plurality of computing units, including: after receiving the query transaction request carrying a query vector sent by the external device, using the memory controller to convert the query transaction request into a memory transaction request and a memory computing request, and performing address mapping on the memory transaction request and the memory computing request respectively to obtain a first memory address corresponding to the memory transaction request and a second memory address corresponding to the memory computing request; and using the computing unit corresponding to the second memory address as the target computing unit.

[0010] In one embodiment, before querying multiple target cluster center vectors from a first mapping table in multiple storage units based on a query vector using a target computing unit, the method further includes: using a memory controller to perform command conversion on a memory transaction request and a memory computing request respectively, to obtain at least one DRAM command corresponding to the memory transaction request and at least one computing command corresponding to the memory computing request; and storing each DRAM command in a DRAM command queue, and storing each computing command in a computing command queue; for each DRAM command in the DRAM command queue, based on a storage unit status table stored in the DRAM, using the memory controller to compare information corresponding to the DRAM command with a storage unit status corresponding to a first memory address; when the information corresponding to the DRAM command matches the storage unit status corresponding to the first memory address, sending the DRAM command to the storage unit corresponding to the first memory address; and using the memory controller to send each computing command in the computing command queue to the computing unit corresponding to the second memory address.

[0011] In one embodiment, the DRAM commands include control commands and arithmetic commands. The control commands include batch activation commands, read commands, and write-back commands; and the arithmetic commands include addition commands, multiplication commands, and table lookup commands.

[0012] In a second aspect, the present application further provides a large-scale data search device for DRAM in-memory computing, which is applied to a memory, the memory including DRAM, the DRAM including multiple storage units and multiple computing units, each storage unit storing a first mapping table and multiple quantized basis vectors, the first mapping table including multiple one-to-one corresponding subvectors and a first cluster center vector, the device including:

[0013] a determination module, configured to determine a target computing unit from a plurality of computing units after receiving a query transaction request carrying a query vector sent by an external device;

[0014] A first query module is configured to query a plurality of target cluster center vectors from a first mapping table in a plurality of storage units based on a query vector using a target computing unit;

[0015] a calculation module, configured to calculate the distance between the query vector and each target cluster center vector using the target calculation unit to obtain a plurality of distance values; and construct a second mapping table based on the plurality of distance values ​​and the plurality of target cluster center vectors, and store the second mapping table in the plurality of storage units;

[0016] The second query module is used to query and obtain R target basis vectors similar to the query vector based on each basis vector and the second mapping table using multiple storage units and target computing units; and send the R target basis vectors to an external device, where R is a positive integer.

[0017] In a third aspect, the present application further provides a memory, the memory including DRAM, the DRAM including multiple storage units and multiple computing units, and the memory is used to implement the steps of any one of the methods in the above-mentioned first aspect.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any one of the methods in the first aspect when executed by a processor.

[0019] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements the steps of any one of the methods in the first aspect when executed by a processor.

[0020] The above-described large-scale data search method, device, memory, readable storage medium, and program product for in-memory computation in DRAM are applied to a memory, including a dynamic random access memory (DRAM), comprising multiple storage units and multiple computing units. Each storage unit stores a first mapping table and multiple quantized basis vectors. The first mapping table includes multiple one-to-one correspondences between subvectors and first cluster center vectors. Upon receiving a query transaction request carrying a query vector from an external device, a target computing unit is determined from the multiple computing units. The target computing unit is then used to retrieve multiple target cluster center vectors from the first mapping table in the multiple storage units based on the query vector. The target computing unit is then used to calculate the distance between the query vector and each target cluster center vector to obtain multiple distance values. A second mapping table is constructed based on the multiple distance values ​​and the multiple target cluster center vectors, and the second mapping table is stored in the multiple storage units. The multiple storage units and the target computing unit are then used to retrieve R target basis vectors similar to the query vector based on each basis vector and the second mapping table. These R target basis vectors are then sent to the external device. By setting up a computing unit in DRAM and completing the search process directly in DRAM, the problem of frequent memory access during the search process in traditional technologies is solved. Therefore, this application can reduce search latency and energy consumption. In addition, through each basis vector and the second mapping table, R target basis vectors similar to the query vector are queried, achieving the goal of obtaining target basis vectors similar to the query vector by table lookup. Compared with the traditional method of obtaining target basis vectors similar to the query vector by comparing and calculating a large amount of high-dimensional vector data, the amount of calculation in the search process is greatly reduced, thereby further reducing search latency and energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of the product quantization ANNS algorithm in the related art;

[0023] Figure 2 It is the schematic diagram of Pyramid-M in the related art;

[0024] Figure 3 1 is a flow chart of a large-scale data search method for DRAM in-memory computing according to one embodiment;

[0025] Figure 4 A schematic diagram of the structure of a DRAM in one embodiment;

[0026] Figure 5 is a schematic structural diagram of a computing unit in one embodiment;

[0027] Figure 6 A schematic diagram of a storage unit table lookup in one embodiment;

[0028] Figure 7 is a schematic structural diagram of a memory in one embodiment;

[0029] Figure 8 is a diagram showing the working principle of a memory controller in one embodiment;

[0030] Figure 9 A schematic diagram of a DRAM command execution process when performing a multiplication operation in one embodiment;

[0031] Figure 10 A structural block diagram of a large-scale data search device for DRAM in-memory computing in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0033] As a key technology in the field of big data search, the Approximate Nearest Neighbor Search (ANNS) algorithm plays an indispensable role in databases containing massive amounts of data. By quickly and efficiently retrieving objects similar to a given query vector from a high-dimensional vector set, it provides key support in various fields, including information retrieval, recommendation systems, and machine learning.

[0034] Among existing technologies, the product quantization (PQ) ANNS algorithm is currently more common. It is a technology for efficient similarity search for high-dimensional data sets. The product quantization ANNS algorithm aims to solve the similarity search problem, that is, the problem of finding nearest neighbors. This is particularly critical in deep learning applications such as image recognition, speech recognition, and natural language processing. Because in these applications, each entity (whether it is an image, speech, or text) can be converted into a high-dimensional embedding vector, and similar entities will be converted into similar vectors. This degree of similarity will be quantified a lot during the search process.

[0035] As an efficient vector quantization method, the product-quantized ANNS algorithm plays a key role in industrial vector indexing applications. The main advantage of the product-quantized ANNS algorithm lies in its ability to effectively partition a high-dimensional space into several subspaces and then perform clustering operations on each of these subspaces, thereby achieving data compression and efficient indexing. Its core advantage lies in its high efficiency, which can significantly reduce the computational and storage requirements for similarity searches on large-scale, high-dimensional datasets.

[0036] Specifically, the product-quantized ANNS algorithm first splits each high-dimensional vector into multiple subvectors. Then, a clustering algorithm (such as K-Means) is independently applied to each subvector to perform quantization. This method maps the original high-dimensional vector into a low-dimensional vector space consisting of multiple cluster center indices. This allows each original high-dimensional vector to be approximated by a set of smaller indices (i.e., cluster center identifiers), significantly reducing storage space requirements and improving search efficiency.

[0037] like Figure 1The figure shows the principle diagram of the product-quantized ANNS algorithm in related art. The entire process of the product-quantized ANNS algorithm includes quantization and search. During the quantization process, the original high-dimensional vector (e.g., an N-dimensional vector) is divided into M groups. The D-dimensional subvectors of each group are clustered using k-means to obtain k cluster center vectors. Each cluster center vector is assigned an n-bit ID, and the original high-dimensional vector is mapped to cluster center vectors with M IDs, thus obtaining the quantized basis vectors. During the search process, a D-dimensional query vector is input and segmented to obtain multiple segment vectors. The distance between each segment vector and the M cluster center vectors is calculated, and the distance values ​​d1, d2, ..., dM are obtained by table lookup. These are aggregated and sorted, and the quantized basis vectors with a Top-R similarity score are output. As you can understand, through quantization, an N × D × 32 vector library can be compressed to an N × M × log2k bit representation, where 32 refers to the dimension of the original high-dimensional vector. For example, when D = 128, M = 4, and k = 256, the compression ratio can reach 128, making it easier to store in memory. This experiment was verified on a CPU system, processing a dataset of 2 billion vectors to ensure search accuracy.

[0038] However, although the ANNS algorithm using product quantization of traditional CPU systems significantly reduces data capacity and significantly improves search efficiency on the original basis, the CPU still needs to perform memory access operations to traverse the vector library stored in memory during each query. During this process, the CPU calculates the distance with each vector and sorts it to determine the nearest neighbor element. Although this method reduces the dimensionality of the search space through quantization, it fails to fundamentally eliminate the latency and energy consumption problems caused by frequent memory access. As the vector space continues to expand, the scalability of this method will face greater challenges, further reducing its performance advantage.

[0039] A search acceleration architecture Pyramid based on hybrid storage structure and in-memory computing has been proposed. The principle of the architecture Pyramid-M of the dynamic random access memory (DRAM) is as follows: Figure 2As shown. The architecture of Pyramid-M includes adjacent dual-inline-memory-modules (DIMMs) and feature DIMMs. At the beginning of the search phase, the host CPU sends the query vector to the memory controller hub (MCH), and the MCH refreshes the priority queue. During each search step, the queue head is read out, and the MCH sends its node index (Vid) to the adjacent DIMM. The adjacent DIMM queries the neighboring indexes (Nids) of Vid and returns them to the MCH. Then multiple Nids are sent to the feature DIMMs of different memory ranks (Rank is a collection of memory chips) for distributed distance calculation. Before the distance calculation, the feature DIMM uses content addressable memory (CAM) to filter the access Nids whose distance to the query vector has been calculated before. Afterwards, the distance calculation is performed on the unaccessed Nids, and the distance is returned to the MCH. The priority queue module in the MCH inserts these distance results into the queue and sorts them. It should be noted that Figure 2 Some unexplained English words are not very relevant to this application. Since they are prior art, they will not be explained in detail again. If you want to understand their principles, you can refer to relevant existing materials.

[0040] However, the Pyramid design faces several challenges. First, there are efficiency challenges arising from its multi-level storage structure. This design introduces two distinct storage hierarchies: memory and hard disk. The information exchange between these two hierarchies incurs significantly higher overhead than traditional data transfers between the CPU and memory. Since hard disk access latency is much higher than memory, this cross-level data transfer not only increases system response time but can also reduce overall performance. Second, the Pyramid design is highly complex in hardware implementation, particularly in DRAM integration. DRAM uses a different manufacturing process than CPUs, and its logic circuits suffer significant performance penalties and come with a significant area overhead. When complex in-memory computation circuits are integrated into DRAM, this additional hardware complexity can significantly increase system overhead, even offsetting the potential performance gains from the integrated computational functionality, negatively impacting overall system performance. Furthermore, the hierarchical graph clustering-based search algorithm employed in the Pyramid design faces significant scalability limitations when working with large datasets. As the amount of data increases, the data structures the algorithm must maintain become increasingly complex, increasing its runtime and potentially rendering it inefficient when processing large datasets. Therefore, when the amount of data reaches a certain level, the scalability of the algorithm will be significantly reduced, becoming a key bottleneck limiting the large-scale application of Pyramid design.

[0041] Based on this, it is necessary to propose effective technical means to solve the above problems. The following specific embodiments are used to describe in detail the technical solution of this application and how the technical solution of this application solves the above technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below in conjunction with the accompanying drawings.

[0042] In one embodiment, Figure 3 As shown, a large-scale data search method for DRAM in-memory computing is provided. The large-scale data search method for DRAM in-memory computing is applied to a memory, which includes a DRAM. The DRAM includes multiple storage units (storage units are represented by "Bank" in all figures of this application) and multiple computing units (computing units are represented by "PIM" in all figures of this application. PIM stands for Processing-In-Memory, which is an in-memory processing technology). Figure 4As shown, a structural diagram of a DRAM is provided; a plurality of storage units correspond one-to-one to a plurality of computing units, each computing unit is arranged in the peripheral area of ​​the corresponding storage unit, each computing unit communicates with each other through a bus, and each computing unit is connected to an input / output (I / O) interface. Each storage unit includes a driving and a reading amplifier, and each storage unit is further finely divided into a plurality of areas, which are respectively used to store a first mapping table, a second mapping table and a plurality of quantized basis vectors; the first mapping table includes a plurality of one-to-one corresponding sub-vectors and a first cluster center vector, the source of which can be found in the quantization process in the above-mentioned product quantized ANNS algorithm principle; the second mapping table includes a plurality of one-to-one corresponding distance values ​​and a target cluster center vector, the source of which can be found in the following steps of a large-scale data search method for in-memory calculations in DRAM; the source of the quantized basis vector can be found in the quantization process in the above-mentioned product quantized ANNS algorithm principle. As shown Figure 5 As shown in the figure, a structural diagram of a computing unit is provided. The computing unit includes a floating-point number (such as FP32) addition unit, a floating-point number (such as FP32) multiplication unit, a table lookup unit, a Top-R queue, an on-chip temporary register Buffer, a crossbar switch CrossBar and other components. Among them, the table lookup unit includes an accumulator and a register Reg.

[0043] The large-scale data search method for DRAM in-memory computing includes the following steps:

[0044] Step 301: After receiving a query transaction request carrying a query vector from an external device, a target computing unit is determined from multiple computing units.

[0045] The external device is such as a host CPU. Determining the target computing unit from the plurality of computing units may be performed according to a scheduling policy.

[0046] Step 302: Utilize a target computing unit to query a plurality of target cluster center vectors from a first mapping table in a plurality of storage units based on a query vector.

[0047] Optionally, the query vector is segmented using a target computing unit to obtain multiple segment vectors; for each segment vector, a sub-vector identical to the segment vector is queried from a first mapping table in multiple storage units to obtain a target cluster center vector corresponding to the segment vector.

[0048] For example, after the query vector is segmented, segment vector 1, segment vector 2, and segment vector 3 are obtained. Subvector 1, subvector 2, and subvector 3 that are identical to segment vector 1, segment vector 2, and segment vector 3 are retrieved from the first mapping table in the multiple storage units. Since the multiple subvectors in the first mapping table correspond one-to-one to the first cluster center vector, the first cluster center vectors corresponding to subvector 1, subvector 2, and subvector 3 can be directly used as the target cluster center vectors for segment vector 1, segment vector 2, and segment vector 3, respectively.

[0049] Step 303: Calculate the distance between the query vector and each target cluster center vector using the target calculation unit to obtain multiple distance values; construct a second mapping table based on the multiple distance values ​​and the multiple target cluster center vectors, and store the second mapping table in multiple storage units.

[0050] The second mapping table includes a plurality of one-to-one corresponding distance values ​​and target cluster center vectors.

[0051] Optionally, the target calculation unit is used to calculate the distance between the query vector and each target cluster center vector to obtain multiple distance values, including: using the target calculation unit to calculate the distance between the query vector and each target cluster center vector by a Euclidean distance algorithm to obtain multiple distance values.

[0052] In the distance query process in step 304, although a single query is relatively simple, the number of basis vectors in the data set may be very large, so the overall time consumption mainly comes from this link, which constitutes a significant part of the system delay. In order to optimize this process, this application adopts a basic reading mechanism with multiple storage units (the multiple storage units can be regarded as DRAM storage units, and the multiple storage units are arranged in a matrix) as the core to simplify the circuit design, and uses each storage unit as a lookup table (LUT) to output the query results, such as Figure 6 As shown in Figure 3, this design method effectively utilizes the storage and access characteristics of DRAM and can improve query efficiency.

[0053] To achieve the above functionality, the second mapping table can be stored in a transposed form in a DRAM memory bank. Specifically, each column of data in the second mapping table is stored in a row of the DRAM memory bank. This storage strategy helps optimize data retrieval and access speed. The column data is the target cluster center vector.

[0054] Step 304 , using multiple storage units and target computing units, based on each basis vector and the second mapping table, query and obtain R target basis vectors similar to the query vector; and send the R target basis vectors to an external device, where R is a positive integer.

[0055] Optionally, the basis vector includes multiple second cluster center vectors; using multiple storage units and target computing units, based on each basis vector and the second mapping table, query to obtain R target basis vectors similar to the query vector, including: for each basis vector, based on the identifier (such as ID) of each second cluster center vector in the basis vector, determine the DRAM row address corresponding to the identifier of each second cluster center vector, based on each DRAM row address, the corresponding storage unit sends the stored distance value to the target computing unit; using the target computing unit, through the Top-R queue, according to the sum of the distance values ​​corresponding to each second cluster center vector in each basis vector, obtain R target basis vectors similar to the query vector.

[0056] In other words, for each basis vector, the ID of each second cluster center vector in each basis vector is used to determine the DRAM row address to be accessed. Subsequently, the data stored in the row is converted into the column address x through a decoding operation. Given the input address x, the distance value dis(x) stored in the corresponding storage unit can be directly output as the query result, such as Figure 6 This approach leverages DRAM's storage structure and access patterns to transform the complex distance calculation and data retrieval process into an efficient storage and table lookup operation. This not only simplifies circuit design but also significantly improves the speed and efficiency of distance queries, greatly reducing latency caused by large-scale data processing and thus optimizing the performance of the overall search system.

[0057] Using the target computation unit and a Top-R queue, R target basis vectors similar to the query vector are obtained based on the sum of the distance values ​​corresponding to the second cluster center vectors in each basis vector. This is achieved as follows: For each basis vector, the distance value corresponding to the second cluster center vector in the basis vector is sent to the target computation unit. The accumulator in the target computation unit merges the distance values ​​corresponding to the second cluster center vectors in the basis vector. The data in the accumulator is then sent to the Top-R queue in the target computation unit. This Top-R queue maintains the top R minimum values ​​in order of magnitude. Through reasonable circuit design, the runtime required to maintain this Top-R queue can be hidden by other operations, thereby optimizing the overall performance of the search system.

[0058] When processing large-scale data sets (e.g., more than 100 million basis vectors), this application distributes and stores the data across multiple storage units. Given the communication challenges between different memory devices, the system adopts an efficient strategy, which is to only transmit the data in the top-R queue back to the external device for final processing. Compared with the traditional system that needs to transmit a large number of basis vectors, this strategy greatly reduces the frequency of data transmission and only needs to transmit k×32 bits of data, achieving the advantages of low latency and low bandwidth consumption. This method not only improves search efficiency, but also significantly reduces the operating cost of the search system.

[0059] The memory includes a dynamic random access memory (DRAM), which includes multiple storage units and multiple computing units. Each storage unit stores a first mapping table and multiple quantized basis vectors. The first mapping table includes multiple one-to-one correspondences between subvectors and first cluster center vectors. After receiving a query transaction request carrying a query vector from an external device, a target computing unit is determined from the multiple computing units. The target computing unit is then used to query multiple target cluster center vectors from the first mapping table in the multiple storage units based on the query vector. The target computing unit is then used to calculate the distance between the query vector and each target cluster center vector to obtain multiple distance values. A second mapping table is constructed based on the multiple distance values ​​and the multiple target cluster center vectors, and the second mapping table is stored in the multiple storage units. The multiple storage units and the target computing unit are then used to query R target basis vectors similar to the query vector based on each basis vector and the second mapping table, and the R target basis vectors are sent to the external device. In this way, by setting the computing unit in the DRAM, the search process is completed directly in the DRAM, solving the problem of frequent memory access during the search process in traditional technologies. Therefore, the present application can reduce search latency and energy consumption. In addition, through each basis vector and the second mapping table, R target basis vectors similar to the query vector are queried, and the target basis vectors similar to the query vector are obtained by table lookup. Compared with the traditional technology of obtaining target basis vectors similar to the query vector through a large amount of high-dimensional vector data comparison calculation, the amount of calculation in the search process is greatly reduced, thereby further reducing the search delay and energy consumption.

[0060] In one embodiment, the memory further includes a memory controller; Figure 7Figure 1 shows a schematic diagram of the memory architecture. The memory controller serves as a bridge between external devices, such as microprocessors (CPUs and GPUs) and I / O devices, and DRAM. The memory controller is responsible for processing memory transaction requests from these external devices, such as query requests. The memory controller provides an arbitration interface and applies a pre-selected storage transaction scheduling policy to determine which initiator of a memory transaction request can submit its memory transaction request to DRAM. This process is called the transaction scheduling phase.

[0061] The second stage is the address mapping stage, also known as address translation. After the memory transaction request passes arbitration, the memory controller will map the memory transaction request to a memory address according to the pre-stored address mapping strategy and convert it into a series of DRAM commands. These command sequences are then stored in the command queue within the memory controller, waiting to be executed. Among them, the address mapping strategy is used to represent the resolution of a given physical address into an index in DRAM, where the index includes the channel ID, memory column Rank ID, storage unit Bank ID, row ID, and column ID. It should be noted that when the runtime behavior of the application does not match the address mapping strategy well, consecutive memory requests in the memory transaction request sequence may be mapped to different rows of the same storage unit in the DRAM array, resulting in storage unit conflicts that will reduce performance. Therefore, the design of the address mapping strategy is also very important.

[0062] Next comes the command scheduling phase. The mapped DRAM commands enter a command queue, where they are scheduled by the memory controller and sent to a valid DRAM address. Command queues can be organized in a variety of flexible ways, ranging from a general queue pool to dedicated queues for each memory cell or column. Based on the DRAM command scheduling policy, commands are dispatched to the DRAM via an electrical signal interface for execution.

[0063] At this point, the memory transaction request sent by the external device has completed the process of instruction decomposition, address conversion, command generation, scheduling, and sending through the memory controller, and then the physical layer interface completes the conversion of DRAM commands into electrical signals.

[0064] The aforementioned memory controller plays a core role in the memory, including address translation, command generation and issuance, and precise timing control. Specifically, at the address translation level, the memory controller is responsible for accurately mapping physical addresses to memory addresses. During the generation and issuance of DRAM commands, the controller is responsible for decoding memory requests into specific DRAM command sequences. Furthermore, the physical layer (PHY layer) within the memory controller further translates the memory controller's commands into underlying signals to drive the DRAM's physical input and output interfaces. Notably, this design does not require any specific modifications to the Double Data Rate Synchronous DRAM (DDR) PHY layer, as this application does not change the DRAM I / O signaling standards. However, given the unique nature of in-memory computing, which requires performing computational tasks within traditional memory space, the memory controller design requires corresponding updates. To ensure smooth communication between the dedicated architecture and external devices, adjustments were made to the memory controller's logic responsible for command generation, scheduling, and transmission. The following is a detailed explanation of the memory controller design for memories equipped with computing units:

[0065] like Figure 8 As shown, a working principle diagram of a memory controller is provided. Upon receiving a query transaction request carrying a query vector from an external device, a target computing unit is determined from multiple computing units. The method includes: upon receiving the query transaction request carrying the query vector from the external device, using the memory controller to convert the query transaction request into a memory transaction request and a memory computing request, and performing address mapping on the memory transaction request and the memory computing request, respectively, to obtain a first memory address corresponding to the memory transaction request and a second memory address corresponding to the memory computing request; and determining the computing unit corresponding to the second memory address as the target computing unit.

[0066] Among them, the memory controller is used to convert the query transaction request into a memory transaction request and a memory calculation request, and then the memory transaction request and the memory calculation request are sent to the DRAM request queue and the calculation request queue (also called the PIM request queue) respectively.

[0067] Address mapping is performed on the memory transaction request and the memory computing request respectively, including: address mapping the physical address carried in the memory transaction request based on the address mapping strategy; address mapping the physical address carried in the memory computing request based on the address mapping strategy.

[0068] Continue to refer Figure 8As shown, before querying multiple target cluster center vectors from a first mapping table in multiple storage units based on a query vector using a target computing unit, the method also includes: using a memory controller to perform command conversion on a memory transaction request and a memory computing request respectively, to obtain at least one DRAM command corresponding to the memory transaction request and at least one computing command corresponding to the memory computing request; and storing each DRAM command in a DRAM command queue, and storing each computing command in a computing command queue; for each DRAM command in the DRAM command queue, based on a storage unit status table stored in the DRAM, using a memory controller to compare information corresponding to the DRAM command with a storage unit status corresponding to a first memory address; when the information corresponding to the DRAM command matches the storage unit status corresponding to the first memory address, the DRAM command is sent to the storage unit corresponding to the first memory address; and using the memory controller to send each computing command in the computing command queue to a computing unit corresponding to a second memory address.

[0069] The memory controller includes a DRAM command generator and a computing command generator (also called a PIM command generator). The memory controller is used to perform command conversion on a memory transaction request and a memory computing request, respectively, to obtain at least one DRAM command corresponding to the memory transaction request and at least one computing command corresponding to the memory computing request, including: using the DRAM command generator to perform command conversion on the memory transaction request to obtain at least one DRAM command; and using the computing command generator to perform command conversion on the memory computing request to obtain at least one computing command corresponding to the memory computing request.

[0070] The memory controller also includes a command arbiter; for each DRAM command in the DRAM command queue, based on the storage cell status table stored in the DRAM, the memory controller is used to compare the information corresponding to the DRAM command with the storage cell status corresponding to the first memory address, including: the command arbiter obtains the storage cell status table from the DRAM, and the storage cell status table includes the status of each storage cell, such as idle, row activation, precharge and other states; the command arbiter compares the information corresponding to the DRAM command with the storage cell status corresponding to the first memory address to determine which DRAM commands are consistent with the status of the storage cell corresponding to the first memory address.

[0071] If a DRAM command has the highest priority in the DRAM command queue and the state of the storage unit corresponding to the first memory address allows the DRAM command to be sent, the DRAM command is sent to the storage unit corresponding to the first memory address. The information corresponding to the DRAM command is such as activation, write, etc.

[0072] After the DRAM command is sent to the storage unit corresponding to the first memory address and to the computation unit corresponding to the second memory address, the storage unit and computation unit execute the corresponding commands, obtaining R target basis vectors similar to the query vector. The specific execution process has been detailed above and will not be repeated here. The R target basis vectors are then sent to a return request queue (also called a request response queue) to be sent to the host side, i.e., the external device, via the host interface.

[0073] Compute command operations have the ability to be executed in parallel on multiple storage units, such as Figure 8 As shown in the lower right corner of the .

[0074] In addition to the dedicated design of the memory controller, the interface information format between external devices and memory, namely the command stream, also needs to be modified to ensure that PIM-related data transmitted by the CPU can be accurately received and processed. DRAM commands, as an abstraction of a series of signal combinations, govern the dynamic behavior of multiple memory cells in DRAM. Similar to the functions of instructions in the CPU's software and hardware interfaces, DRAM commands generally follow a fixed set of types, such as READ, WRITE, ACTIVATE, PRECHARGE, etc., and are presented in the form of a truth table. In addition, the RFU (Reserved for Future Use) command in DRAM provides a certain degree of flexibility to the system, allowing for a certain degree of subsequent programming based on needs to achieve specific functional implementations.

[0075] As shown in Table 1 below, DRAM commands are mainly divided into three parts: the first is the original basic DRAM commands, which ensure the normal execution of the original DRAM storage function; the second is some control commands required to ensure the normal execution of in-memory computing functions, such as the BACT command that supports the activation of multiple storage units. Their function is to control the movement of data between various levels of nodes in DRAM to prepare for computing operations; the third is the core arithmetic commands, which are responsible for executing calculations in DRAM. They include addition commands, multiplication commands, and table lookup commands commonly used in search algorithms. These commands do not have the function of controlling data movement and therefore have completely different properties from the original DRAM commands.

[0076] Table 1

[0077]

[0078] like Figure 9The following figure shows a schematic diagram of the DRAM command execution flow during a multiplication operation, illustrating the various DRAM commands and their timing relationships. Once the WRITEIN command is detected on the command bus, the DRAM begins writing data to its scratchpad buffer. Subsequently, to prepare for the upcoming arithmetic operation, the memory controller issues a BATCHACT command to activate multiple memory cells.

[0079] Next, after receiving the MUL command, the DRAM performs the multiplication calculation in parallel in the corresponding computational units of each memory cell. This step is crucial for in-memory computation, significantly improving computational efficiency through parallel processing. Finally, coordinated by the WRITEBACK command, the DRAM writes the processed results back to the corresponding memory cells. A certain time interval must be maintained between two consecutive commands. This interval represents the minimum time span that must elapse between the completion of one command and the start of the next. This time interval is set to ensure correct data processing and smooth command execution, avoiding potential data conflicts or processing errors.

[0080] The implementation of in-memory computing in DRAM and its stringent command timing requirements demonstrate the necessity of expanding the DRAM command set and the need to precisely control the timing of commands when designing in-memory computing to support DRAM, ensuring correct computation and data accuracy.

[0081] In summary, this application optimizes search algorithm performance in large-scale vector retrieval scenarios. Specifically, it addresses algorithmic bottlenecks by designing a dedicated in-DRAM computing solution. This solution incorporates a unique in-DRAM computing architecture (data path) and memory controller design. This design effectively reduces frequent memory accesses during the search process, significantly alleviating performance latency and reducing power consumption, while ensuring functional completeness.

[0082] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0083] Based on the same inventive concept, embodiments of the present application also provide a large-scale data search device for in-DRAM computing, which is used to implement the aforementioned large-scale data search method for in-DRAM computing. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the large-scale data search device for in-DRAM computing provided below can be found in the above-mentioned limitations of the large-scale data search method for in-DRAM computing, and will not be repeated here.

[0084] In an exemplary embodiment, Figure 10 As shown, a large-scale data search device for DRAM in-memory calculation is provided, which is applied to a memory, wherein the memory includes a DRAM, and the DRAM includes multiple storage units and multiple calculation units. Each storage unit stores a first mapping table and multiple quantized basis vectors. The first mapping table includes multiple one-to-one corresponding subvectors and a first cluster center vector. The large-scale data search device 100 for DRAM in-memory calculation includes: a determination module 1001, a first query module 1002, a calculation module 1003, and a second query module 1004, wherein:

[0085] A determination module 1001 is configured to determine a target computing unit from a plurality of computing units after receiving a query transaction request carrying a query vector sent by an external device;

[0086] A first query module 1002 is configured to query a plurality of target cluster center vectors from a first mapping table in a plurality of storage units based on a query vector using a target computing unit;

[0087] A calculation module 1003 is configured to calculate the distance between the query vector and each target cluster center vector using a target calculation unit to obtain a plurality of distance values; construct a second mapping table based on the plurality of distance values ​​and the plurality of target cluster center vectors, and store the second mapping table in a plurality of storage units;

[0088] The second query module 1004 is used to query and obtain R target basis vectors similar to the query vector based on each basis vector and the second mapping table using multiple storage units and target computing units; and send the R target basis vectors to an external device, where R is a positive integer.

[0089] In one embodiment, the first query module 1002 is specifically configured to segment the query vector using a target computing unit to obtain a plurality of segment vectors; and for each segment vector, query a subvector identical to the segment vector from a first mapping table in a plurality of storage units to obtain a target cluster center vector corresponding to the segment vector.

[0090] In one embodiment, the basis vector includes multiple second cluster center vectors; the second query module 1004, specifically for each basis vector, determines the DRAM row address corresponding to the identifier of each second cluster center vector based on the identifier of each second cluster center vector in the basis vector, and based on each DRAM row address, the corresponding storage unit sends the stored distance value to the target computing unit; using the target computing unit, through the Top-R queue, according to the sum of the distance values ​​corresponding to each second cluster center vector in each basis vector, obtains R target basis vectors similar to the query vector.

[0091] In one embodiment, the memory further includes a memory controller; the device further includes a first conversion module, which is configured to, after receiving a query transaction request carrying a query vector sent by an external device, use the memory controller to convert the query transaction request into a memory transaction request and a memory computing request, and perform address mapping on the memory transaction request and the memory computing request respectively to obtain a first memory address corresponding to the memory transaction request and a second memory address corresponding to the memory computing request; and use the computing unit corresponding to the second memory address as the target computing unit.

[0092] In one embodiment, the device further includes a second conversion module, which is used to use a memory controller to perform command conversion on a memory transaction request and a memory computing request respectively, to obtain at least one DRAM command corresponding to the memory transaction request and at least one computing command corresponding to the memory computing request; and store each DRAM command in a DRAM command queue, and store each computing command in a computing command queue; for each DRAM command in the DRAM command queue, based on a storage unit status table stored in the DRAM, use the memory controller to compare information corresponding to the DRAM command with a storage unit status corresponding to a first memory address; when the information corresponding to the DRAM command matches the storage unit status corresponding to the first memory address, the DRAM command is sent to the storage unit corresponding to the first memory address; and use the memory controller to send each computing command in the computing command queue to a computing unit corresponding to a second memory address.

[0093] In one embodiment, the DRAM commands include control commands and arithmetic commands. The control commands include batch activation commands, read commands, and write-back commands; and the arithmetic commands include addition commands, multiplication commands, and table lookup commands.

[0094] Each module in the aforementioned large-scale data search device for in-memory computing using DRAM can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0095] In an exemplary embodiment, a memory is provided, the memory including DRAM, the DRAM including multiple storage units and multiple computing units, and the memory is used to implement the steps of the method described in any of the above-mentioned embodiments of the large-scale data search method for DRAM in-memory computing.

[0096] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any of the above-mentioned large-scale data search method embodiments for DRAM in-memory computing are implemented.

[0097] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the method described in any of the above-mentioned embodiments of the large-scale data search method for DRAM in-memory computing.

[0098] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0099] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0100] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A large-scale data search method for DRAM in-memory computing, characterized in that: Applied to a memory, the memory including a dynamic random access memory (DRAM), the DRAM including multiple storage units and multiple computing units, each of the storage units storing a first mapping table and multiple quantized basis vectors, the first mapping table including multiple one-to-one corresponding subvectors and a first cluster center vector, the basis vector being composed of cluster center vectors corresponding to the multiple subvectors; The multiple sub-vectors are obtained by segmenting the high-dimensional vector; the method includes: After receiving a query transaction request carrying a query vector sent by an external device, determining a target computing unit from the multiple computing units; Utilizing the target calculation unit, querying a plurality of target cluster center vectors from the first mapping tables in the plurality of storage units based on the query vector; Utilizing the target calculation unit to calculate the distance between the query vector and each of the target cluster center vectors to obtain a plurality of distance values; and constructing a second mapping table based on the plurality of distance values ​​and the plurality of target cluster center vectors, and storing the second mapping table in the plurality of storage units; Utilizing the multiple storage units and the target calculation unit, based on each of the basis vectors and the second mapping table, query and obtain R target basis vectors similar to the query vector; and sending the R target basis vectors to the external device, where R is a positive integer.

2. The method according to claim 1, characterized in that The step of using the target calculation unit to query a plurality of target cluster center vectors from the first mapping tables in the plurality of storage units based on the query vector includes: Using the target computing unit, segmenting the query vector to obtain a plurality of segment vectors; For each segment vector, a sub-vector identical to the segment vector is searched from the first mapping table in the plurality of storage units to obtain the target cluster center vector corresponding to the segment vector.

3. The method according to claim 1, characterized in that The basis vectors include a plurality of second cluster center vectors; and the querying and obtaining R target basis vectors similar to the query vector based on each of the basis vectors and the second mapping table using the plurality of storage units and the target calculation unit includes: For each of the basis vectors, based on the identifier of each of the second cluster center vectors in the basis vectors, determining a DRAM row address corresponding to the identifier of each of the second cluster center vectors, and based on each of the DRAM row addresses, each of the corresponding storage units sends the stored distance value to the target computing unit; The target calculation unit is used to obtain R target basis vectors similar to the query vector through a Top-R queue according to the sum of the distance values ​​corresponding to the second cluster center vectors in the basis vectors.

4. The method according to claim 1, wherein The memory further includes a memory controller; after receiving a query transaction request carrying a query vector sent by an external device, determining a target computing unit from the multiple computing units includes: After receiving a query transaction request carrying a query vector sent by the external device, converting the query transaction request into a memory transaction request and a memory computing request using the memory controller, and performing address mapping on the memory transaction request and the memory computing request respectively to obtain a first memory address corresponding to the memory transaction request and a second memory address corresponding to the memory computing request; The computing unit corresponding to the second memory address is used as the target computing unit.

5. The method according to claim 4, characterized in that Before querying a plurality of target cluster center vectors from the first mapping tables in the plurality of storage units based on the query vector using the target calculation unit, the method further includes: Utilizing the memory controller, performing command conversion on the memory transaction request and the memory computing request respectively to obtain at least one DRAM command corresponding to the memory transaction request and at least one computing command corresponding to the memory computing request; storing each of the DRAM commands in a DRAM command queue, and storing each of the computing commands in a computing command queue; For each DRAM command in the DRAM command queue, based on a storage unit status table stored in the DRAM, using the memory controller to compare information corresponding to the DRAM command with a storage unit status corresponding to the first memory address, and when the information corresponding to the DRAM command matches the storage unit status corresponding to the first memory address, sending the DRAM command to the storage unit corresponding to the first memory address; The memory controller is used to send each of the computing commands in the computing command queue to the computing unit corresponding to the second memory address.

6. The method according to claim 5, characterized in that The DRAM commands include control commands and arithmetic commands. The control commands include batch activation commands, read commands, and write-back commands; the arithmetic commands include addition commands, multiplication commands, and table lookup commands.

7. A large-scale data search device for DRAM in-memory computing, characterized in that: Applied to a memory, the memory comprising a DRAM, the DRAM comprising a plurality of storage units and a plurality of computing units, each of the storage units storing a first mapping table and a plurality of quantized basis vectors, the first mapping table comprising a plurality of one-to-one corresponding subvectors and a first cluster center vector, the basis vector being composed of cluster center vectors corresponding to the plurality of subvectors; The multiple sub-vectors are obtained by segmenting the high-dimensional vector; the device includes: a determination module, configured to determine a target computing unit from the plurality of computing units after receiving a query transaction request carrying a query vector sent by an external device; a first query module, configured to query a plurality of target cluster center vectors from the first mapping tables in the plurality of storage units based on the query vector using the target calculation unit; a calculation module, configured to calculate the distance between the query vector and each of the target cluster center vectors using the target calculation unit to obtain a plurality of distance values; and construct a second mapping table based on the plurality of distance values ​​and the plurality of target cluster center vectors, and store the second mapping table in the plurality of storage units; A second query module is used to use the multiple storage units and the target computing unit to query and obtain R target basis vectors similar to the query vector based on each basis vector and the second mapping table; and send the R target basis vectors to the external device, where R is a positive integer.

8. A memory, characterized in that: The memory comprises a DRAM, which comprises a plurality of storage units and a plurality of calculation units. The memory is used to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images