Hybrid storage progressive precision writing and retrieval method and apparatus

Abstract

The application discloses a hybrid storage progressive precision writing and retrieval method and device, wherein the method comprises the following steps: quantizing a current real value vector of a target online model to a target bit width, determining an initial base value vector corresponding to the current real value vector of the target bit width, and writing the initial base value vector into a preset resistive random access memory; based on the current real value vector of the target bit width and the initial base value vector, calculating an initial residual error vector corresponding thereto, and performing a preset residual error quantization encoding operation on the initial residual error vector to obtain a residual error code word; performing a preset bit plane decomposition and block organization processing operation on the residual error code word to obtain a plurality of residual error bit vector blocks, and continuously storing the residual error bit vector blocks into a dynamic random access memory; and based on the initial base value vector and the initial residual error vector, performing a preset dynamic precision retrieval operation on a current query vector to obtain a corresponding retrieval result, so that the dynamic balance of retrieval performance and precision is realized.

Description

Technical Field

[0001] This application relates to the field of hybrid storage writing and retrieval technology, and in particular to a hybrid storage progressive precision writing and retrieval method and apparatus. Background Technology

[0002] To improve the performance and energy efficiency of large-scale vector retrieval, the industry has gradually adopted near-memory computing / in-memory computing acceleration architecture based on RRAM (Resistive Random Access Memory). Vector data is stored in multi-bit form in an RRAM crossbar (i.e., a crossbar array of resistive random access memory), and vector inner product or similarity calculations are achieved by accumulating the conductance within the array. However, when RRAM is applied to a complex scenario of "high-frequency streaming vector updates, multi-bit storage, and online retrieval," existing technologies reveal the following key problems:

[0003] 1. The retrieval latency for new vectors is too high: (1) Multi-bit RRAM cells are usually finely programmed using methods such as incremental step pulse programming (ISPP); as the target number of bits in the cell increases from 1 bit to 8 bits, the number of pulse rounds and verify times required for writing increases significantly, and the programming delay of 8 bits is much higher than that of 1 bit.

[0004] (2) In existing random number generation, in order to store a new vector in RRAM with full 8-bit precision and participate in the calculation, it is often necessary to wait until the entire round of multi-bit programming is completed before adding the vector to the index. In business scenarios where vectors are continuously flowing in, this means that the new vector is in an "unretrieval" state for a long time, or can only be temporarily stored in high-cost volatile storage (DRAM (Dynamic Random Access Memory) / SRAM (Static Random Access Memory)), which cannot meet the needs of online business for real-time or near real-time updates.

[0005] 2. The write bandwidth pressure and peak power consumption issues caused by multi-bit RRAM writing: (1) When the vector stream continues to arrive and each new vector immediately performs a complete 8-bit ISPP programming, a large number of write operations are highly concentrated in time, which brings huge write bandwidth pressure and instantaneous power consumption peaks to the RRAM array and peripheral circuits.

[0006] (2) The write operation shares physical array and peripheral resources with the read / compute operation of the foreground, which can easily lead to contention, resulting in increased retrieval latency jitter and decreased service stability.

[0007] (3) The existing solution lacks a mechanism that can separate multi-bit writing from the critical path in the foreground and smoothly distribute the multi-bit writing process to the background idle time and local idle array area without affecting the online retrieval service, thereby reducing the writing peak and balancing the system load.

[0008] 3. DRAM capacity and power consumption waste issues in RRAM-DRAM hybrid architecture (1) To compensate for the slow writing speed of RRAM, some solutions choose to temporarily store the new vector or its high-precision copy in DRAM / SRAM and then synchronize it to RRAM when conditions permit.

[0009] (2) If there is a lack of fine precision management and state convergence mechanism, vector data in DRAM will often exist for a long time, resulting in DRAM capacity being continuously occupied by a large amount of temporary data, which is not easy to reclaim; the same vector has redundant copies in RRAM and DRAM, resulting in a double waste of storage space and access energy consumption.

[0010] 4. The accuracy status is disconnected from the retrieval process, making it impossible to perform accuracy-adaptive retrieval control in response to streaming updates: (1) Traditional RRAM vector retrieval accelerators are usually designed with the assumption that once a vector is stored in RRAM, its accuracy is consistent and fixed, and the retrieval hardware also assumes that all vectors have the same bit width during operation.

[0011] (2) When faced with the situation of "continuous insertion of vector stream and multi-bit programming not yet completed", the existing architecture lacks a mechanism to bind the "current written precision state" with the "retrieval stage / strategy": it cannot use the written 1-2 bit state to perform coarse sorting and filtering in advance; it cannot flexibly choose to use only RRAM coarse value or combine DRAM residual for fine sorting according to the current precision state of different vectors.

[0012] In summary, existing RRAM vector retrieval systems generally suffer from problems such as "high latency in retrieving new vectors, high peak pressure for multi-bit writes, excessive DRAM usage, and a disconnect between accuracy and retrieval process" when facing real-world business scenarios involving high-frequency streaming vector updates. These issues make it difficult for the system to achieve a dynamic balance between streaming updates, query latency, retrieval accuracy, and storage overhead, and urgently need to be addressed. Summary of the Invention

[0013] This application provides a hybrid storage progressive precision writing and retrieval method and apparatus to solve problems such as high latency in new vector retrieval, high peak pressure of multi-bit writing, excessive DRAM usage, and disconnect between precision and retrieval process in existing RRAM vector retrieval systems in real-world scenarios with high-frequency streaming vector updates.

[0014] The first aspect of this application provides a hybrid storage progressive precision writing and retrieval method, comprising the following steps: quantizing the current real-value vector of a target online model to a target bit width, determining the initial base value vector corresponding to the current real-value vector of the target bit width, and writing the initial base value vector into a preset resistive random access memory; calculating the corresponding initial residual vector based on the current real-value vector of the target bit width and the initial base value vector, and performing a preset residual quantization encoding operation on the initial residual vector to obtain the corresponding residual codeword; performing a preset bit-plane decomposition and block organization processing operation on the residual codeword to obtain multiple residual bit vector blocks, and storing the multiple residual bit vector blocks consecutively in a dynamic random access memory; and performing a preset dynamic precision retrieval operation on the current query vector of the target online model based on the initial base value vector and the initial residual vector to obtain the corresponding retrieval result.

[0015] A second aspect of this application provides a hybrid storage asymptotic precision writing and retrieval device, comprising: a writing module, configured to quantize the current real-value vector of a target online model to a target bit width, determine an initial base value vector corresponding to the current real-value vector of the target bit width, and write the initial base value vector into a preset resistive random access memory; an encoding module, configured to calculate a corresponding initial residual vector based on the current real-value vector of the target bit width and the initial base value vector, and perform a preset residual quantization encoding operation on the initial residual vector to obtain a corresponding residual codeword; and a retrieval module, configured to perform a preset bit-plane decomposition and block organization processing operation on the residual codeword to obtain multiple residual bit vector blocks, and continuously store the multiple residual bit vector blocks into a dynamic random access memory, and perform a preset dynamic precision retrieval operation on the current query vector of the target online model based on the initial base value vector and the initial residual vector to obtain a corresponding retrieval result.

[0016] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hybrid storage progressive precision write and retrieval method as described in the above embodiments.

[0017] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the hybrid storage progressive precision write and retrieval method described above.

[0018] A fifth aspect of this application provides a computer program product, including a computer program that is executed to implement the above-described hybrid storage progressive precision write and retrieval method.

[0019] Therefore, the embodiments of this application have the following beneficial effects: The embodiments of this application can be implemented by quantizing the current real-valued vector of the target online model to the target bit width, determining the initial base value vector corresponding to the current real-valued vector of the target bit width, and writing the initial base value vector into a preset resistive switching memory; based on the current real-valued vector of the target bit width and the initial base value vector, calculating the corresponding initial residual vector, and performing a preset residual quantization encoding operation on the initial residual vector to obtain the corresponding residual codeword; performing a preset bit-plane decomposition and block organization processing operation on the residual codeword to obtain multiple residual bit vector blocks, and storing the multiple residual bit vector blocks continuously in a dynamic random access memory; and based on the initial base value vector and the initial residual vector, performing a preset dynamic precision retrieval operation on the current query vector of the target online model to obtain the corresponding retrieval result. This application uses bit-plane partitioning and residual quantization as the core organizational units, organically combining the processes of "rapidly forming coarse-level base values ​​in RRAM, storing residuals in DRAM for accurate reconstruction, performing adaptive retrieval based on accuracy and residual state, and progressively improving accuracy in the background using fine-grained ISPP and conductivity distribution convergence criteria while releasing DRAM block by block." This provides a low-latency, high-energy-efficiency, accuracy-controllable, and easily long-term-manageable data representation and writing scheme for multi-bit RRAM applications in streaming vector retrieval scenarios. Therefore, it solves the problems of high latency for new vector retrieval, high peak pressure for multi-bit writes, excessive DRAM usage, and the disconnect between accuracy and the retrieval process in existing RRAM vector retrieval systems in high-frequency streaming vector update scenarios.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a hybrid storage progressive precision write and retrieval method provided according to an embodiment of this application; Figure 2This is an example diagram of a hybrid storage progressive precision write and retrieval apparatus according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0022] Among them, 10-hybrid storage progressive precision writing and retrieval device; 100-writing module, 200-encoding module, 300-retrieval module; 301-memory, 302-processor, 303-communication interface. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0024] The hybrid storage progressive precision writing and retrieval method and apparatus of this application are described below with reference to the accompanying drawings. Addressing the problems mentioned in the background art, this application provides a hybrid storage progressive precision writing and retrieval method. In this method, the current real-valued vector of the target online model is quantized to a target bit width, and an initial base value vector corresponding to the current real-valued vector of the target bit width is determined and written into a preset resistive random access memory (RRAM). Based on the current real-valued vector of the target bit width and the initial base value vector, the corresponding initial residual vector is calculated, and a preset residual quantization encoding operation is performed on the initial residual vector to obtain the corresponding residual codeword. A preset bit-plane decomposition and block organization processing operation is performed on the residual codeword to obtain multiple residual bit vector blocks, and the multiple residual bit vector blocks are continuously stored in a dynamic random access memory (DRAM). Based on the initial base value vector and the initial residual vector, a preset dynamic precision retrieval operation is performed on the current query vector of the target online model to obtain the corresponding retrieval result. This application uses bit-plane partitioning and residual quantization as the core organizational units, organically combining the processes of "rapidly forming coarse-level base values ​​in RRAM, storing residuals in DRAM for accurate reconstruction, performing adaptive retrieval based on accuracy and residual state, and progressively improving accuracy in the background using fine-grained ISPP and conductivity distribution convergence criteria while releasing DRAM block by block." This provides a low-latency, high-energy-efficiency, accuracy-controllable, and easily long-term-manageable data representation and writing scheme for multi-bit RRAM applications in streaming vector retrieval scenarios. Therefore, it solves the problems of high latency for new vector retrieval, high peak pressure for multi-bit writes, excessive DRAM usage, and the disconnect between accuracy and the retrieval process in existing RRAM vector retrieval systems in high-frequency streaming vector update scenarios.

[0025] Specifically, Figure 1This is a flowchart illustrating a hybrid storage progressive precision writing and retrieval method provided in an embodiment of this application.

[0026] like Figure 1 As shown, this hybrid storage progressive precision write and retrieval method includes the following steps: In step S101, the current real value vector of the target online model is quantized to the target bit width, and the initial base value vector corresponding to the current real value vector of the target bit width is determined and written into the preset resistive variable memory.

[0027] In step S102, based on the current real value vector and the initial base value vector of the target bit width, the corresponding initial residual vector is calculated, and the initial residual vector is subjected to a preset residual quantization encoding operation to obtain the corresponding residual codeword.

[0028] In the embodiments of this application, each RRAM cell supports multiple conductance states, corresponding to several discrete numerical levels, which can represent analog values ​​with finite precision, such as 1 to 8 bits. At any given time, the current conductance of a cell can be interpreted as a "base value approximation" of the target digital value, which gradually approaches the target value with subsequent write rounds. Furthermore, each dimension of the vector is mapped to the rows and columns of the crossbar according to fixed rules. During querying, embodiments of this application can apply a corresponding voltage to the crossbar array to achieve large-scale parallel inner product or similar operations between the vector and the stored vector.

[0029] Secondly, the DRAM in this application embodiment does not store "unwritten low-order bits" in a simple sense, but stores the codeword (i.e., residual codeword) after quantization of the numerical difference (i.e., the initial residual vector) between the target 8-bit quantization vector (i.e., the current real value vector) and the current RRAM base value (i.e., the initial base value vector).

[0030] It is understood that the residual codewords in the embodiments of this application can be encoded by scalar encoding by dimension or by subvector. The bits after residual encoding can be stored in DRAM by subsequent bit-plane and fixed-width block method, so as to restore high-precision representation in the fine sorting stage and update or release after writing in the background.

[0031] Optionally, in one embodiment of this application, quantizing the current real-valued vector of the target online model to the target bit width, determining the initial base value vector corresponding to the current real-valued vector of the target bit width, and writing the initial base value vector into a preset resistive random access memory (RRAM) includes: quantizing each dimension of the current real-valued vector into a target bit fixed-point value based on a preset quantization rule to obtain a target digital vector of the target bit width corresponding to the current real-valued vector; determining the write latency, device resolution, and system data entry latency requirements of the RRAM, selecting an initial precision level based on the write latency, device resolution, and system data entry latency requirements, and mapping the target digital vector to the precision space corresponding to the initial precision level to obtain the initial base value vector corresponding to the current real-valued vector; and performing a preset write operation on the coarse conductance state corresponding to the initial base value vector based on a preset incremental step pulse programming mechanism to write the initial base value vector into the RRAM and record the precision state corresponding to the current real-valued vector.

[0032] It should be noted that when the online model generates a new real-valued vector (i.e., the current real-valued vector, for example, with a dimension of 512), the embodiments of this application can perform the following operations on the current real-valued vector: 1. Vector quantization to target bit width: (1) Quantize each dimension of the real value vector into a b-bit fixed-point value according to a pre-defined quantization rule (e.g., linear or logarithmic scale), for example, b=8.

[0033] (2) The quantized result can be regarded as the target 8-bit digital vector (i.e., the target digital vector), which is the value that this application embodiment ultimately hopes to converge in RRAM.

[0034] 2. Select the initial RRAM programming precision level and generate a base value approximation: (1) According to the RRAM write delay, device resolution and system requirements for data entry delay, an initial precision level can be selected, denoted as precision_level = p0. For example, p0 can be 1 or 2, which means that the cell only needs to distinguish 2 to the power of p0 conductivity levels.

[0035] (2) In the embodiments of this application, the target 8-bit vector can be projected to a coarser precision space to obtain a base value approximation vector (i.e., the initial base value vector). For example, only the highest 1 bit or 2 bits can be retained, or the bit value can be mapped to a representative value of several predefined conductance intervals. The representative value vector is the initial base value vector to be written into RRAM.

[0036] 3. Fast writing of RRAM base value in the foreground: (1) The embodiments of this application may use the ISPP mechanism to perform fast write operations only for the coarse conductance state that approximates the base value: 1) Each unit requires only a small number of pulses and verification steps to achieve a coarse-grained conductivity range; 2) The write latency is significantly lower than the multiple rounds of fine programming required to directly write a full 8 bits.

[0037] (2) After writing, the embodiment of this application can record the precision level of the vector as precision_level = p0, and initialize the relevant conductivity distribution statistics. At this time, the vector can participate in the coarse sorting calculation operation by relying on the coarse approximation in RRAM.

[0038] 4. Calculate the initial residual and write it to DRAM: (1) Subtract the base approximation vector just written to RRAM from the target 8-bit vector (subtraction within the same numerical domain) to obtain an initial residual vector.

[0039] (2) Perform residual quantization encoding on this residual vector: 1) In this embodiment of the application, signed or unsigned amplitude encoding can be performed using b_r bits (e.g., 3 or 4 bits) for each dimension of the residual; 2) In the embodiments of this application, several dimensions can also be combined into sub-vectors and sub-vector residual quantization can be used.

[0040] (3) In this embodiment, the residual encoded bits can be written into DRAM in a block layout according to bit-plane combined with fixed vector width, and the residual_level can be initialized to b_r (or its corresponding quantization level).

[0041] (4) From this moment on, in any situation where it is necessary to recover an approximate 8-bit precision representation, the system can reconstruct a vector representation close to the target 8 bits by “reading the base approximation from RRAM, reading the residual codeword from DRAM and decoding it”.

[0042] Those skilled in the art should understand that RRAM cells can achieve multi-bit storage by adjusting the conductance through the application of set / reset pulses of varying numbers and amplitudes. However, multi-bit writing requires multiple rounds of fine-grained ISPP, and the write latency and energy consumption increase significantly with the target precision. Secondly, the ISPP programming mechanism gradually approximates the target conductance value through a cycle of "short pulse-verification-adjustment." Since each round can only slightly adjust the conductance, it is very suitable for the mechanism of "gradually approximating the target conductance using a brief idle window" in this application embodiment. Furthermore, when representing vectors, this application embodiment can treat the current value in the RRAM as a base value, and then quantize the residual difference between the base value and the target quantized value. The residual gradually decreases with each write operation, which is the theoretical basis for the bidirectional convergence of "RRAM precision improvement - DRAM information reduction" in this application embodiment.

[0043] It should be noted that, for continuously arriving vector streams, the embodiments of this application can immediately write the new vector into the RRAM with a coarse-precision base value and store its residual codeword relative to the target multi-bit representation in the DRAM, so that the new vector can be retrieved by the online vector library with extremely low write latency, while ensuring that a high-precision representation with a uniform bit width can be recovered at any write stage through "RRAM base value and DRAM residual".

[0044] Furthermore, in the embodiments of this application, the system may choose to directly store the remaining unwritten low-order bits of the original RRAM into DRAM in a conventional manner, and then concatenate them with the written portion of RRAM to form a complete vector during the fine sorting stage. However, this scheme cannot handle analog errors during the RRAM writing process, cannot provide a guarantee of reconstruction accuracy, and the DRAM read / write granularity is irregular, resulting in low efficiency.

[0045] It is understood that, through the rapid writing of coarse-level base values ​​to the RRAM, new vectors can immediately enter the retrieval path without waiting for complete multi-bit programming, maintaining millisecond or even microsecond-level data entry latency even under high-frequency streaming input. Secondly, through the "RRAM base value-DRAM residual" joint representation mechanism, even if the vector is only in the early write state, it can still recover approximately 8-bit precision during the fine sorting stage, ensuring the final accuracy and stability of online retrieval, unaffected by online updates, and supporting gradual approximation to the final high precision. Furthermore, by decomposing multi-bit ISPP programming into multiple rounds of fine-grained approximation operations and executing them on query gaps and locally idle arrays, the write process and retrieval task do not interfere with each other, achieving both low latency in the foreground and high precision in the background. In addition, as the RRAM conductance approaches the target, the residual quantization value continuously decreases and is reclaimed block by block, allowing DRAM to only briefly store intermediate states when necessary and release them after precision convergence, avoiding long-term occupation and fragmentation problems.

[0046] In step S103, the residual codeword is subjected to preset bit plane decomposition and block organization processing to obtain multiple residual bit vector blocks, and the multiple residual bit vector blocks are continuously stored in dynamic random access memory. Based on the initial base value vector and the initial residual vector, the current query vector of the target online model is subjected to preset dynamic precision retrieval operation to obtain the corresponding retrieval result.

[0047] Furthermore, embodiments of this application may maintain metadata fields for each vector, such as: vector ID; precision_level refers to the current distinguishable conductance level or equivalent number of effective bits on the RRAM side, for example, a value of 1 to 8; residual_level refers to the number of effective bits or quantization level corresponding to the current DRAM residual codeword, for example, 0 indicates no residual, and a positive integer indicates the residual strength; the starting address, block size, and number of blocks of the DRAM residual bit-plane; the most recent insertion time and the timestamp of the most recent background update; and statistical information serving the conductance distribution convergence criterion (such as the mean, standard deviation, and estimated distance between peaks of the corresponding cell conductance).

[0048] Secondly, when the query vector arrives, the embodiments of this application can use the retrieval engine to determine the processing path of each vector in the coarse and fine ranking stages based on the precision_level and residual_level of each vector: in the coarse ranking stage, the current base value of RRAM is used as much as possible to complete the fast filtering without accessing DRAM; in the fine ranking stage, the DRAM residual is read as needed and combined with the RRAM base value to restore a vector representation with a precision close to the target 8-bit precision.

[0049] Optionally, in one embodiment of this application, the residual codeword is subjected to preset bit plane decomposition and block organization processing operations to obtain multiple residual bit vector blocks, and the multiple residual bit vector blocks are continuously stored in dynamic random access memory, including: decomposing the residual codeword into multiple residual bit planes, and dividing the vector dimension corresponding to each residual bit plane into residual bit vector blocks of fixed width; storing the multiple residual bit vector blocks continuously in dynamic random access memory according to the residual bit plane and block index order, and when accessing dynamic random access memory, obtaining at least one residual bit vector block corresponding to the residual bit plane in dynamic random access memory.

[0050] In actual implementation, to avoid fragmented access, difficulty in updating, and difficulty in releasing residual information as a whole in DRAM, this application embodiment can store residual codewords in the DRAM layer using a dedicated layout method that divides them into blocks according to bit-plane and fixed vector width (wherein, bit-plane storage layout refers to splicing the same bit of multiple vectors (or the same bit-plane in the residual) into a continuous block with a fixed width, which can significantly improve the efficiency of DRAM burst access and enable DRAM space to be released in blocks after background overwrite), as described in detail below: 1. Residual bit-plane decomposition and block organization: (1) Assuming that each dimension of residual encoding uses b_r bits, for example 4 bits, the residual can be decomposed into b_r residual bit planes (each plane corresponds to one bit).

[0051] (2) For each residual bit plane, the embodiments of this application can divide the vector dimension into tiles of fixed width (i.e., residual bit vector blocks), for example, each 64-dimensional or 128-dimensional block: 1) Taking a 512-dimensional vector as an example, for a certain residual bit plane, the embodiments of this application can combine dimensions 0-63 into the first block, 64-127 into the second block, and so on.

[0052] (3) Each residual bit vector block consists of a set of continuous residual bits, called a residual bit vector block, which has a fixed bit length and facilitates burst read and write of DRAM.

[0053] 2. DRAM contiguous storage and block-level access: (1) In the embodiments of this application, the residual bit vector blocks can be stored continuously in DRAM according to the residual bit plane and block index order. The specific arrangement can be sorted by vector ID or physical address.

[0054] (2) When updating a certain precision level of a vector in the background, the controller can read out or overwrite one or more blocks on a certain residual bit plane through a small number of DRAM burst accesses, instead of performing a large number of scattered scalar accesses.

[0055] 3. Residual shrinkage and block-level release: (1) As the background RRAM programming proceeds, the base value approximation vector in the RRAM gradually approaches the target 8-bit value, and the magnitude of the corresponding residual vector will become smaller and smaller.

[0056] (2) After residual quantization, there may be some blocks on a certain residual bit plane where all the encoded bits are 0, or all residual amplitudes are lower than the preset threshold. At this time, the block is no longer meaningful for the recovery accuracy, and the corresponding residual can be considered to have converged.

[0057] (3) In this embodiment of the application, marking and releasing are performed on a unit basis: residual bit vector blocks. 1) Mark these blocks as "purgeable" in the metadata; 2) The residual_level can be reduced as needed (e.g., by reducing the number of effective residual bits). 3) The DRAM space management module can reclaim these blocks as a whole and redistribute them from the free list for use by subsequent newly inserted vectors.

[0058] Therefore, through the above layout and management method, the DRAM only retains the residual information that is currently needed to correct the RRAM base value error. This information will naturally decrease and eventually disappear as the RRAM accuracy improves, thereby avoiding the waste and fragmentation of DRAM space caused by storing scattered residuals for a long time.

[0059] It should be noted that the embodiments of this application may also use traditional block storage based on sub-vectors instead of bit-plane block storage. That is to say, the embodiments of this application may store vector slices in DRAM according to fixed dimensions (such as 16 / 32 dimensions) instead of organizing them according to bit-plane. However, this method will result in only updating the conductance corresponding to some dimensions during the overwrite, which will not achieve consistent bit-plane asymptotic convergence and will increase the randomness of DRAM access.

[0060] Therefore, the embodiments of this application store residual codewords in blocks of bit-plane and fixed-dimensional tiles (such as 64 / 128 dimensions) and use these blocks as the basic unit for background updates and releases. This allows the system to efficiently read and write residuals through burst access even at high update rates, and automatically reclaim DRAM space block by block as RRAM precision improves. Thus, the embodiments of this application can transform data access from scattered scalar operations to whole-block burst transmissions, reducing the number of DRAM random accesses, improving throughput, and reducing energy consumption and access conflicts during the rewrite phase.

[0061] Optionally, in one embodiment of this application, based on the initial base value vector and the initial residual vector, a preset dynamic precision retrieval operation is performed on the current query vector of the target online model to obtain the corresponding retrieval result. This includes: quantizing the current query vector to obtain a query quantized vector that meets a preset compatibility requirement with the initial base value vectors stored in the resistive random access memory (RRAM); calculating the similarity between the query quantized vector and all initial base value vectors stored in the RRAM based on the initial base value vectors; performing a preset coarse ranking operation on all initial base value vectors according to the similarity to obtain a corresponding candidate base value vector set; and determining the vector precision level and vector residual level of each candidate base value vector in the candidate base value vector set; and determining whether the vector precision level of the current candidate base value vector reaches the target precision level. The system first checks the target accuracy and whether the vector residual level is a preset value. If the vector accuracy level of the current candidate base value vector reaches the target accuracy and the vector residual level is a preset value, then a preset similarity calculation operation is performed based on the conductance value of the resistive random access memory and the query quantization vector. If the vector residual level of the current candidate base value vector is not a preset value, then the current base value vector corresponding to the current candidate base value vector is read from the resistive random access memory, and the residual codeword block is read from the dynamic random access memory according to the bit plane block. Based on the preset residual decoding rules, the residual codeword block is restored to the corresponding numerical residual, and the numerical residual is added to the current base value vector to obtain the recovered vector that meets the preset accuracy requirements. Then, a preset fine ranking calculation operation is performed on the recovered vector and the current query vector to obtain the retrieval result.

[0062] In the specific implementation process, in order to balance real-time searchability and high-precision results in the streaming update scenario, the embodiment of this application divides the retrieval into two stages: coarse ranking and fine ranking, which are jointly controlled by the vector's precision_level and residual_level. The specific process is as follows: 1. Coarse-sorting stage: Relying solely on RRAM base values ​​for approximation: (1) Before the current query vector enters the RRAM array, it will also be quantized to obtain a representation compatible with the storage vector (both coarse and high precision versions can be constructed at the same time).

[0063] (2) In the RRAM crossbar, the system can use the currently stored base values ​​to approximate and calculate the rough similarity between the query vector and all vectors in the library (or vectors in the candidate clusters): 1) For vectors with low precision (e.g., precision_level = 1 or 2), a large number of dissimilar vectors can be filtered out quickly and at extremely low cost by using approximate inner product based on sign or a few conductance levels or similar Hamming distance calculations. 2) For vectors with higher precision, more conductance levels can be used to provide a more refined coarse rating.

[0064] (3) The coarse sorting stage relies entirely on RRAM and does not access DRAM, thus minimizing DRAM bandwidth consumption and access latency.

[0065] 2. Fine-tuning stage: High-precision representation reconstructed jointly from RRAM base values ​​and DRAM residuals: (1) After the coarse sorting is completed, a candidate set (e.g., hundreds to thousands of vectors) is selected and then proceeded to the fine sorting.

[0066] (2) For each vector in the candidate set: 1) If the precision_level of the vector has reached the target precision (e.g., 8 bits) and the residual_level is 0, it means that its complete numerical information has been fully stored in RRAM and there is no need to access DRAM; at this time, high-precision similarity calculation can be performed directly based on the RRAM conductance and the query vector. 2) If a residual still exists (residual_level greater than 0), this embodiment can read the current base value approximation of the vector from RRAM; read the residual codeword block from DRAM in bit-plane blocks according to the address given by the metadata; restore the residual codeword to the numerical residual according to the predetermined residual decoding rules, and add it to the base value approximation to recover a high-precision vector close to the target 8 bits; use the recovered high-precision vector and the query vector to perform fine sorting calculation (such as precise inner product or distance).

[0067] 3. Adaptive fine-ranking strategy based on accuracy status: (1) The search engine can adaptively adjust the fine ranking strategy according to the precision_level and residual_level of different vectors, for example: 1) For vectors with high precision_level and very small residual_level, RRAM values ​​can be used directly or only a small amount of residual can be read, saving DRAM access; 2) For vectors with a low precision level but a large residual level, the residual is only read when high-precision sorting is required. If it is only in the coarse sorting stage, the DRAM can be temporarily ignored. 3) When DRAM bandwidth is tight, vectors with high precision_level can be prioritized for fine sorting to achieve better bandwidth utilization efficiency.

[0068] Therefore, through the coarse and fine sorting operations described above, the embodiments of this application can effectively ensure that newly inserted vectors that have only completed the initial base value writing can immediately participate in coarse sorting without waiting for the complete 8-bit writing to be completed. Secondly, if high-precision results are required, the embodiments of this application can still restore approximately 8-bit precision through DRAM residuals during the fine sorting stage. In addition, as the background progressive writing proceeds, the distinguishing ability of coarse sorting naturally improves, the dependence of fine sorting on DRAM gradually weakens, and ultimately most vectors can complete high-precision retrieval by relying only on RRAM.

[0069] It should be noted that in the embodiments of this application, background writes can be triggered only based on QPS or pipeline idle conditions, but do not detect the local array unused situation generated by query routing. This scheme is simple to implement, but it is difficult to effectively utilize the crossbar local idle window, the background write speed is reduced and the DRAM residual release cycle is longer.

[0070] In summary, the embodiments of this application can maintain precision_level and residual_level fields for each vector. In the coarse sorting stage, it relies only on the current base value in RRAM for fast filtering. In the fine sorting stage, it reads the residual from DRAM as needed to reconstruct a high-precision vector, realizing dynamic precision retrieval that can be retrieved even if the writing is not completed, and automatically improving the accuracy of the retrieval results over time.

[0071] It is understood that, as the precision level increases, the coarse-sorting capability is enhanced and the candidate set is reduced, thereby reducing the amount of DRAM residual reads and enabling the system to achieve higher retrieval QPS and more stable latency distribution over long-term operation. Furthermore, by identifying unused crossbar subarrays in the current query and performing background writes in these areas, the embodiments of this application can significantly improve the write progress without interrupting the retrieval process, accelerating high-precision vector convergence.

[0072] Optionally, in one embodiment of this application, after performing a preset dynamic precision retrieval operation on the current query vector of the target online model to obtain the corresponding retrieval result, the method further includes: performing a periodic scanning operation on the vector metadata and residual metadata corresponding to the current real-value vector and the initial residual vector to calculate the corresponding precision level and residual level, and obtaining a vector set that meets the preset precision improvement requirements based on the precision level and residual level; performing multiple rounds of incremental step pulse programming writing operations on the vector set according to the residual bit vector block order, and calculating the preset dynamic resistive memory unit associated with each residual bit vector block after the multiple rounds of incremental step pulse programming writing. The system performs a read operation to obtain multiple conductance distribution indices. In response to multiple conductance distribution indices meeting preset conductance convergence requirements, it determines that the conductance state corresponding to the current residual bit vector block meets preset accuracy requirements and recalculates the residual vector of the associated dimension of the previous residual bit vector block. It then performs a preset residual encoding quantization operation on the residual vector to obtain the corresponding new residual codeword. If the residual size corresponding to the residual vector is greater than a preset residual threshold, the corresponding residual bit vector block in the dynamic random access memory is accessed using the new residual codeword; otherwise, the corresponding residual bit vector block in the dynamic random access memory is marked as a reclaimable block and the memory is released.

[0073] It should be noted that, in the embodiments of this application, the controller can monitor the system load, query QPS, total DRAM residual, accuracy status of each vector, and convergence of conductance distribution; in query gaps or crossbar subarray regions not visited by the current query, multiple rounds of ISPP writes are scheduled in the background to make small adjustments to the RRAM base value and gradually improve the accuracy level; in addition, after each round of writing, the residual is recalculated and written back to DRAM, or the DRAM space is released in blocks when the residual converges to zero.

[0074] As a feasible approach, to gradually converge the RRAM conductance states of all vectors from coarse precision to the target multi-bit precision without significantly interfering with the foreground retrieval performance, this application implements a background progressive multi-bit writing, residual update, and DRAM management mechanism. Using residual bit-plane blocks as the basic operation unit, multi-bit programming is split into multiple short-lived idle windows and local idle arrays for execution. The specific process is as follows: 1. Scheduling strategy for background precision enhancement controller: (1) The controller periodically scans the vector metadata and residual metadata and selects the vector set whose accuracy needs to be improved. For example, if the precision_level is less than the target precision (e.g., 8 bits), and the residual_level is greater than a certain threshold, it means that the residual is still large; the vector insertion time has been too long and the accuracy has not yet converged.

[0075] (2) When selecting a specific update target, the controller needs to comprehensively consider: whether the current total DRAM residual is close to the set high water level threshold; the current query throughput and RRAM / DRAM bandwidth utilization; the frequency and priority of the vector recently retrieved; and whether the standard deviation, peak spacing, etc. of the corresponding RRAM cell conductance distribution have approached convergence.

[0076] (3) For systems that use inverted file index (IVF) and other structures, the controller can identify the crossbar rows / columns that are not used in the current round of queries based on the cluster set hit by the current batch query, treat these areas as "local idle arrays", and prioritize scheduling background writes and residual updates on them.

[0077] 2. ISPP writing and conductance convergence determination based on residual bit-plane blocks: (1) For the selected vector and its precision level to be improved, the controller processes it in units of residual bit-plane blocks: 1) First, in this embodiment of the application, the corresponding residual bit-plane block can be read from the DRAM in a burst manner according to the DRAM layout, and the current conductance value of the corresponding cell can be read from the RRAM at the same time; 2) Then, according to the residual magnitude and direction, this embodiment can calculate the approximate number or amplitude of set or reset pulses that need to be applied to each cell in this round, so that the conductivity value is slightly closer to the target range; 3) Each round only adjusts a limited number of units and controls the adjustment range, so that the writing process is fine-grained and gradual in terms of time and conductivity space, without causing significant interference to the front-end calculation.

[0078] (2) After multiple rounds of writing, the embodiments of this application can quickly read and test the RRAM cells associated with the block and statistically analyze relevant indicators of conductivity distribution, such as: whether the mean conductivity is close to the expected target range; whether the standard deviation of conductivity is lower than the preset convergence threshold; whether the distance between the distribution peaks corresponding to different conductivity levels is large enough to indicate that each level is clearly distinguishable.

[0079] (3) When the above indicators meet the preset conditions, the embodiment of this application can determine that the conductivity state corresponding to the block has reached the required accuracy, and can proceed to the next step of residual recalculation and DRAM block-level processing.

[0080] 3. Residual recalculation and DRAM block-level recycling: (1) After the conductance corresponding to a certain residual bit-plane block has reached a new level of precision, the system recalculates the residual vector of the associated dimension based on the value represented by the latest RRAM conductance.

[0081] (2) Quantize the new residual vector again using the same residual coding method to obtain a new residual codeword: 1) If the residual is still significant (e.g., exceeding the minimum scale of the residual quantization used), then overwrite the corresponding block in DRAM with the new residual codeword and update the residual_level; 2) If the residuals are all zero or the absolute value is lower than the preset threshold, the block has no substantial value for restoring high precision. It can be marked as "recyclable" in DRAM management and released as a whole for subsequent use.

[0082] (3) As the write-detect-residual update cycle is executed repeatedly, the conductance state in the RRAM gradually approaches the target 8-bit digital value, and the corresponding residual codewords become increasingly sparse and the bit width becomes smaller and smaller until they eventually disappear completely. At this point, the vector can be marked as being completely represented by the high precision of the RRAM, and subsequent retrieval will no longer access the DRAM, providing higher energy efficiency for long-term system operation.

[0083] Those skilled in the art will understand that in vector retrieval, a query typically only accesses a small number of clusters. Therefore, most rows / columns in RRAM stored across clusters are not accessed in this round of queries. This application utilizes this characteristic to allow these localized free areas to be used for background writing, improving write throughput. Secondly, the multi-bit state of an RRAM cell can be viewed as a combination of several conductance peaks. This application uses a decrease in standard deviation and an increase in peak spacing for convergence judgment, thereby ensuring that the physical conductance stably falls within the correct range, improving storage reliability.

[0084] It should be noted that the embodiments of this application can use a multi-bit write method with a fixed write step size without performing conductance distribution convergence criterion detection. That is, the background write can choose to apply a fixed number of pulses each time without detecting peak separation or standard deviation. However, this scheme is prone to overprogramming or underprogramming, making it difficult to stably reduce the residual and affecting the consistency of the final vector accuracy. Secondly, the embodiments of this application can also replace the incremental write mechanism with periodic full updates. In other words, in the embodiments of this application, the system can also periodically pause part of the query and complete a high-precision write of all vectors to be updated at once. However, this design strategy will cause large retrieval latency jitter and high energy consumption, which cannot meet the requirements of continuous and high-frequency streaming update scenarios.

[0085] Therefore, this application embodiment accelerates multi-bit convergence of streaming vectors without significantly interfering with online retrieval latency by scheduling multiple rounds of incremental ISPP programming with a small number of pulses on the crossbar subarrays that are not currently accessed during the query gap and in the current inverted index. It gradually improves the RRAM accuracy by using convergence criteria such as the standard deviation of conductance distribution and peak separation. Furthermore, it recalculates the residuals and reclaims DRAM at the block level after each round of writing. It is understood that the embodiments of this application can use the standard deviation of conductivity distribution and peak separation as convergence indicators to ensure that the conductivity state is controllable and close to the target distribution after each round of writing, thereby improving the stability of multi-bit programming and the quality of vector representation. Secondly, since most of the fine sorting calculations occur on vectors that are completely filled with bits, while background writing and DRAM access gradually decrease, the long-term operating power consumption is significantly reduced, making it more suitable for large-scale embedded libraries and real-time recommendation, search and other scenarios.

[0086] In summary, considering the characteristics of multi-bit RRAM write latency increasing significantly with the target number of bits and the fact that intermediate conductance states can only approximate the target digital level, in scenarios with a continuous inflow of new vectors, the embodiments of this application decompose the write process into the following two paths: 1. Fast path in the foreground: Quickly write a small number of key bits or coarse conductance intervals into RRAM to form a base value approximation that can be used immediately for calculation; 2. Background progressive path: Utilizing query gaps and local idle arrays, the target conductance is gradually approximated with multiple rounds of short pulses. The difference between the current RRAM base value and the target 8-bit value is quantized and stored in DRAM, and gradually compressed until it is cleared to zero as the conductance accuracy increases.

[0087] Unlike simply placing unwritten low-order bits directly into DRAM, this application explicitly assumes that at any write stage, the RRAM stores a base value approximation of the final 8-bit target fixed-point value (which can be considered an approximate value corresponding to the current physical conductance), while the DRAM stores the residual between this base value approximation and the target 8-bit value, after quantization and encoding. The DRAM side adopts a layout combining bit-plane and fixed vector width blocks to organize the residual information into bit vector blocks that facilitate burst access and overall reclamation. Each vector maintains a precision_level field and a residual_level field, ensuring that a uniform bit-width (e.g., 8-bit) numerical representation can be recovered through "RRAM base value combined with DRAM residual" at any RRAM write stage, thereby supporting the system behavior of "retrieval even when writing is incomplete".

[0088] Through the above mechanism, the embodiments of this application can effectively enable the RRAM array to participate in the retrieval of new vectors with extremely low write latency. Furthermore, the background can use multiple rounds of ISPP to progressively improve the conductivity accuracy, gradually reduce the residual, and release DRAM in blocks. In addition, the retrieval process can adaptively select the coarse-to-fine data path according to the accuracy and residual status, so as to achieve a dynamic balance between performance and accuracy.

[0089] According to the hybrid storage progressive precision writing and retrieval method proposed in this application, the current real-value vector of the target online model is quantized to the target bit width, and the initial base value vector corresponding to the current real-value vector of the target bit width is determined and written into a preset resistive switching memory. Based on the current real-value vector of the target bit width and the initial base value vector, the corresponding initial residual vector is calculated, and a preset residual quantization encoding operation is performed on the initial residual vector to obtain the corresponding residual codeword. The residual codeword is subjected to a preset bit-plane decomposition and block organization processing operation to obtain multiple residual bit vector blocks, and the multiple residual bit vector blocks are continuously stored in dynamic random access memory. Based on the initial base value vector and the initial residual vector, a preset dynamic precision retrieval operation is performed on the current query vector of the target online model to obtain the corresponding retrieval result. This application uses bit-plane block partitioning and residual quantization as the core organizational units, organically combining the processes of "rapidly forming coarse-level base values ​​in RRAM, storing residuals in DRAM for accurate reconstruction, performing adaptive retrieval based on accuracy and residual state, and gradually improving accuracy in the background and releasing DRAM block by block using fine-grained ISPP and conductivity distribution convergence criteria." This provides a low-latency, high-energy-efficiency, accuracy-controllable, and easy-to-manage data representation and writing scheme for the application of multi-bit RRAM in streaming vector retrieval scenarios.

[0090] Secondly, the hybrid storage progressive precision write and retrieval apparatus according to embodiments of this application is described with reference to the accompanying drawings.

[0091] Figure 2 This is a block diagram of a hybrid storage progressive precision write and retrieval device according to an embodiment of this application.

[0092] like Figure 2 As shown, the hybrid storage progressive precision write and retrieval device 10 includes: a write module 100, an encoding module 200, and a retrieval module 300.

[0093] The writing module 100 is used to quantize the current real value vector of the target online model to the target bit width, determine the initial base value vector corresponding to the current real value vector of the target bit width, and write the initial base value vector into a preset resistive switching memory. The encoding module 200 is used to calculate the corresponding initial residual vector based on the current real value vector and the initial base value vector of the target bit width, and to perform a preset residual quantization encoding operation on the initial residual vector to obtain the corresponding residual codeword. The retrieval module 300 is used to perform preset bit plane decomposition and block organization processing on the residual codeword to obtain multiple residual bit vector blocks, and to store the multiple residual bit vector blocks continuously in the dynamic random access memory. Based on the initial base value vector and the initial residual vector, it performs a preset dynamic precision retrieval operation on the current query vector of the target online model to obtain the corresponding retrieval results.

[0094] Optionally, in one embodiment of this application, the hybrid storage progressive precision write and retrieval device 10 of this application embodiment further includes: a first calculation module, a read and test module, a second calculation module, and a release module.

[0095] The first calculation module is used to perform a preset dynamic precision retrieval operation on the current query vector of the target online model to obtain the corresponding retrieval results, and then perform a periodic scanning operation on the vector metadata and residual metadata corresponding to the current real value vector and the initial residual vector to calculate the corresponding precision level and residual level, and obtain a set of vectors that meet the preset precision improvement requirements based on the precision level and residual level.

[0096] The read and test module is used to perform multiple rounds of incremental step pulse programming and writing operations on the vector set according to the residual bit vector block sequence, and to calculate the preset read and test operations of the resistive random access memory unit associated with each residual bit vector block after the multiple rounds of incremental step pulse programming and writing, so as to obtain multiple conductance distribution indicators.

[0097] The second calculation module is used to determine whether the conductance state corresponding to the current residual bit vector block meets the preset accuracy requirement when multiple conductance distribution indicators meet the preset conductance convergence requirements, and to recalculate the residual vector of the associated dimension of the previous residual bit vector block.

[0098] The release module is used to perform a preset residual encoding quantization operation on the residual vector to obtain the corresponding new residual codeword. If the residual size corresponding to the residual vector is greater than the preset residual threshold, the corresponding residual bit vector block in the dynamic random access memory is dynamically retrieved using the new residual codeword. Otherwise, the corresponding residual bit vector block in the dynamic random access memory is marked as a reclaimable block and the memory is released.

[0099] Optionally, in one embodiment of this application, the writing module 100 includes: a first quantization unit, a selection unit, and a recording unit.

[0100] The first quantization unit is used to quantize each dimension of the current real-valued vector into a target bit fixed-point value based on a preset quantization rule, so as to obtain a target digital vector with a target bit width corresponding to the current real-valued vector.

[0101] The selection unit is used to determine the write latency, device resolution, and system data entry latency requirements of the resistive random access memory. Based on the write latency, device resolution, and system data entry latency requirements, an initial precision level is selected, and the target digital vector is mapped to the precision space corresponding to the initial precision level to obtain the initial base value vector corresponding to the current real value vector.

[0102] The recording unit is used to perform a preset write operation on the coarse conductance state corresponding to the initial base value vector based on a preset incremental step pulse programming mechanism, so as to write the initial base value vector into the resistive variable memory and record the precision state corresponding to the current real value vector.

[0103] Optionally, in one embodiment of this application, the retrieval module 300 includes a decomposition unit and a storage unit.

[0104] The decomposition unit is used to decompose the residual codeword into multiple residual bit planes and divide the vector dimension corresponding to each residual bit plane into a fixed-width residual bit vector block.

[0105] The storage unit is used to store multiple residual bit vector blocks consecutively in the dynamic random access memory according to the residual bit plane and block index order, and to retrieve at least one residual bit vector block corresponding to the residual bit plane in the dynamic random access memory when accessing the dynamic random access memory.

[0106] Optionally, in one embodiment of this application, the retrieval module 300 further includes: a second quantization unit, a coarse ranking unit, a judgment unit, a similarity calculation unit, a reading unit, and a fine ranking unit.

[0107] The second quantization unit is used to perform quantization operations on the current query vector to obtain a query quantized vector that meets the preset compatibility requirements with the initial base value vector stored in the resistive random access memory, and to calculate the similarity between the query quantized vector and all the initial base value vectors stored in the resistive random access memory based on the initial base value vector.

[0108] The coarse-ranking unit is used to perform a pre-defined coarse-ranking operation on all initial base value vectors based on similarity to obtain the corresponding candidate base value vector set, and to determine the vector precision level and vector residual level of each candidate base value vector in the candidate base value vector set.

[0109] The judgment unit is used to determine whether the vector precision level of the current candidate base value vector has reached the target precision and whether the vector residual level is a preset value.

[0110] The similarity calculation unit is used to perform a preset similarity calculation operation based on the conductance value of the resistive switching memory and the query quantization vector if the vector precision level of the current candidate base value vector reaches the target precision and the vector residual level is a preset value.

[0111] The read unit is used to read the current base value vector corresponding to the current candidate base value vector from the resistive random access memory and read the residual codeword block from the dynamic random access memory according to the bit plane block if the vector residual level of the current candidate base value vector is not a preset value.

[0112] The fine sorting unit is used to restore the residual codeword block to the corresponding numerical residual based on the preset residual decoding rules, and add the numerical residual to the current base value vector to obtain the recovery vector that meets the preset accuracy requirements. The recovery vector is then used to perform a preset fine sorting calculation operation with the current query vector to obtain the retrieval result.

[0113] It should be noted that the foregoing explanation of the hybrid storage progressive precision write and retrieval method embodiment also applies to the hybrid storage progressive precision write and retrieval device of this embodiment, and will not be repeated here.

[0114] The hybrid storage progressive precision writing and retrieval device proposed in the embodiments of this application includes a writing module 100, used to quantize the current real-value vector of the target online model to the target bit width, determine the initial base value vector corresponding to the current real-value vector of the target bit width, and write the initial base value vector into a preset resistive random access memory; an encoding module 200, used to calculate the corresponding initial residual vector based on the current real-value vector of the target bit width and the initial base value vector, and perform a preset residual quantization encoding operation on the initial residual vector to obtain the corresponding residual codeword; and a retrieval module 300, used to perform a preset bit-plane decomposition and block organization processing operation on the residual codeword to obtain multiple residual bit vector blocks, and continuously store the multiple residual bit vector blocks into a dynamic random access memory, and perform a preset dynamic precision retrieval operation on the current query vector of the target online model based on the initial base value vector and the initial residual vector to obtain the corresponding retrieval result. This application uses bit-plane block partitioning and residual quantization as the core organizational units, organically combining the processes of "rapidly forming coarse-level base values ​​in RRAM, storing residuals in DRAM for accurate reconstruction, performing adaptive retrieval based on accuracy and residual state, and gradually improving accuracy in the background and releasing DRAM block by block using fine-grained ISPP and conductivity distribution convergence criteria." This provides a low-latency, high-energy-efficiency, accuracy-controllable, and easy-to-manage data representation and writing scheme for the application of multi-bit RRAM in streaming vector retrieval scenarios.

[0115] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 301, the processor 302, and the computer program stored on the memory 301 and capable of running on the processor 302.

[0116] When the processor 302 executes the program, it implements the hybrid storage progressive precision write and retrieval method provided in the above embodiments.

[0117] Furthermore, electronic devices also include: Communication interface 303 is used for communication between memory 301 and processor 302.

[0118] The memory 301 is used to store computer programs that can run on the processor 302.

[0119] The memory 301 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0120] If the memory 301, processor 302, and communication interface 303 are implemented independently, then the communication interface 303, memory 301, and processor 302 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0121] Optionally, in a specific implementation, if the memory 301, processor 302, and communication interface 303 are integrated on a single chip, then the memory 301, processor 302, and communication interface 303 can communicate with each other through an internal interface.

[0122] Processor 302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0123] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described hybrid storage progressive precision writing and retrieval method.

[0124] This application also provides a computer program product, including a computer program, which, when executed, is used to implement the above-described hybrid storage progressive precision write and retrieval method.

[0125] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0126] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0127] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0128] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0129] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0130] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0131] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0132] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A hybrid storage progressive precision writing and retrieval method, characterized in that, Includes the following steps: The current real-valued vector of the target online model is quantized to the target bit width, and the initial base value vector corresponding to the current real-valued vector of the target bit width is determined, and the initial base value vector is written into a preset resistive switching memory. Based on the current real value vector of the target bit width and the initial base value vector, calculate the corresponding initial residual vector, and perform a preset residual quantization encoding operation on the initial residual vector to obtain the corresponding residual codeword; The residual codeword is subjected to preset bit plane decomposition and block organization processing operations to obtain multiple residual bit vector blocks, and the multiple residual bit vector blocks are continuously stored in dynamic random access memory. Based on the initial base value vector and the initial residual vector, a preset dynamic precision retrieval operation is performed on the current query vector of the target online model to obtain the corresponding retrieval result.

2. The hybrid storage progressive precision writing and retrieval method according to claim 1, characterized in that, After performing a preset dynamic precision retrieval operation on the current query vector of the target online model to obtain the corresponding retrieval results, the method further includes: Perform periodic scanning operations on the vector metadata and residual metadata corresponding to the current real value vector and the initial residual vector to calculate the corresponding precision level and residual level, and obtain a set of vectors that meet the preset precision improvement requirements based on the precision level and the residual level. The vector set is subjected to multiple rounds of incremental step pulse programming and writing operations in the order of residual bit vector blocks, and the resistive switching memory cell associated with each residual bit vector block is calculated and a preset read and test operation is performed to obtain multiple conductance distribution indices. In response to the multiple conductance distribution indices meeting the preset conductance convergence requirements, it is determined that the conductance state corresponding to the current residual bit vector block meets the preset accuracy requirements, and the residual vector of the associated dimension of the previous residual bit vector block is recalculated. A preset residual encoding quantization operation is performed on the residual vector to obtain a corresponding new residual codeword. If the residual size corresponding to the residual vector is greater than a preset residual threshold, the corresponding residual bit vector block in the dynamic random access memory is used with the new residual codeword. Otherwise, the corresponding residual bit vector block in the dynamic random access memory is marked as a reclaimable block and memory is released.

3. The hybrid storage progressive precision writing and retrieval method according to claim 1, characterized in that, The step of quantizing the current real-valued vector of the target online model to the target bit width, determining the initial base value vector corresponding to the current real-valued vector of the target bit width, and writing the initial base value vector into a preset resistive switching memory includes: Based on a preset quantization rule, each dimension of the current real-valued vector is quantized into a target bit fixed-point value to obtain a target digital vector with a target bit width corresponding to the current real-valued vector; The write latency, device resolution, and system data entry latency requirements of the resistive random access memory are determined. Based on the write latency, device resolution, and system data entry latency requirements, an initial precision level is selected, and the target digital vector is mapped to the precision space corresponding to the initial precision level to obtain the initial base value vector corresponding to the current real value vector. Based on a preset incremental step pulse programming mechanism, a preset write operation is performed on the coarse conductance state corresponding to the initial base value vector to write the initial base value vector into the resistive variable memory and record the precision state corresponding to the current real value vector.

4. The hybrid storage progressive precision writing and retrieval method according to claim 3, characterized in that, The step of performing a preset bit-plane decomposition and block organization operation on the residual codeword to obtain multiple residual bit vector blocks, and then storing the multiple residual bit vector blocks consecutively in the dynamic random access memory, includes: The residual codeword is decomposed into multiple residual bit planes, and the vector dimension corresponding to each residual bit plane is divided into residual bit vector blocks of fixed width. The plurality of residual bit vector blocks are stored sequentially in the dynamic random access memory according to the residual bit plane and block index order, and when the dynamic random access memory is accessed, at least one residual bit vector block corresponding to the residual bit plane in the dynamic random access memory is obtained.

5. The hybrid storage progressive precision writing and retrieval method according to claim 4, characterized in that, The step of performing a preset dynamic precision retrieval operation on the current query vector of the target online model based on the initial base value vector and the initial residual vector to obtain the corresponding retrieval results includes: The current query vector is quantized to obtain a query quantized vector that meets a preset compatibility requirement with the initial base value vector stored in the resistive switching memory. Based on the initial base value vector, the similarity between the query quantized vector and all the initial base value vectors stored in the resistive switching memory is calculated. Based on the similarity, a pre-defined coarse sorting operation is performed on all the initial base values ​​to obtain the corresponding candidate base value vector set, and the vector precision level and vector residual level of each candidate base value vector in the candidate base value vector set are determined. Determine whether the vector precision level of the current candidate base value vector has reached the target precision, and whether the vector residual level is a preset value; If the vector precision level of the current candidate base value vector reaches the target precision, and the vector residual level is the preset value, then a preset similarity calculation operation is performed based on the conductance value of the resistive switching memory and the query quantization vector. If the vector residual level of the current candidate base value vector is not the preset value, then the current base value vector corresponding to the current candidate base value vector is read from the resistive random access memory, and the residual codeword block is read from the dynamic random access memory according to the bit plane block. Based on the preset residual decoding rules, the residual codeword block is restored to the corresponding numerical residual, and the numerical residual is added to the current base value vector to obtain a recovery vector that meets the preset accuracy requirements. The recovery vector is then used to perform a preset fine sorting calculation operation with the current query vector to obtain the retrieval result.

6. A hybrid storage progressive precision write and retrieval device, characterized in that, include: The writing module is used to quantize the current real-valued vector of the target online model to the target bit width, determine the initial base value vector corresponding to the current real-valued vector of the target bit width, and write the initial base value vector into a preset resistive switching memory. The encoding module is used to calculate the corresponding initial residual vector based on the current real value vector of the target bit width and the initial base value vector, and to perform a preset residual quantization encoding operation on the initial residual vector to obtain the corresponding residual codeword. The retrieval module is used to perform preset bit-plane decomposition and block organization processing on the residual codeword to obtain multiple residual bit vector blocks, and to continuously store the multiple residual bit vector blocks in a dynamic random access memory. Based on the initial base vector and the initial residual vector, it performs a preset dynamic precision retrieval operation on the current query vector of the target online model to obtain the corresponding retrieval results.

7. The hybrid storage progressive precision write and retrieval device according to claim 6, characterized in that, Also includes: The first calculation module is used to perform a preset dynamic precision retrieval operation on the current query vector of the target online model to obtain the corresponding retrieval result, and then perform a periodic scanning operation on the vector metadata and residual metadata corresponding to the current real value vector and the initial residual vector to calculate the corresponding precision level and residual level, and obtain a set of vectors that meet the preset precision improvement requirements based on the precision level and the residual level. The read and test module is used to perform multiple rounds of incremental step pulse programming and writing operations on the vector set according to the residual bit vector block order, and to calculate the preset read and test operations of the resistive switching memory unit associated with each residual bit vector block after the multiple rounds of incremental step pulse programming and writing to obtain multiple conductance distribution indicators. The second calculation module is used to determine that the conductance state corresponding to the current residual bit vector block meets the preset accuracy requirement when the multiple conductance distribution indicators meet the preset conductance convergence requirements, and to recalculate the residual vector of the associated dimension of the previous residual bit vector block. The release module is used to perform a preset residual encoding quantization operation on the residual vector to obtain a corresponding new residual codeword, and to determine whether the residual size corresponding to the residual vector is greater than a preset residual threshold. If so, the new residual codeword is used to store the corresponding residual bit vector block in the dynamic random access memory. Otherwise, the corresponding residual bit vector block in the dynamic random access memory is marked as a reclaimable block and the memory is released.

8. The hybrid storage progressive precision write and retrieval device according to claim 6, characterized in that, The writing module includes: The first quantization unit is used to quantize each dimension of the current real-valued vector into a target bit fixed-point value based on a preset quantization rule, so as to obtain a target digital vector with a target bit width corresponding to the current real-valued vector; The selection unit is used to determine the write latency, device resolution, and system data entry latency requirements of the resistive random access memory, and select an initial precision level based on the write latency, device resolution, and system data entry latency requirements, and map the target digital vector to the precision space corresponding to the initial precision level to obtain the initial base value vector corresponding to the current real value vector. The recording unit is used to perform a preset write operation on the coarse conductance state corresponding to the initial base value vector based on a preset incremental step pulse programming mechanism, so as to write the initial base value vector into the resistive variable memory and record the precision state corresponding to the current real value vector.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the hybrid memory progressive precision write and retrieval method as described in any one of claims 1-5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the hybrid storage progressive precision write and retrieval method as described in any one of claims 1-5.

Images