A hardware-accelerated embedded and vector search integrated SSD storage device for a retrieval-enhanced generation system

Abstract

The application relates to the cross field of SSD storage technology and retrieval enhancement generation technology, in particular to a hardware acceleration embedding and vector retrieval integrated SSD storage device for a retrieval enhancement generation system, which comprises an interface module, a hardware Embedding module, a vector storage module, a hardware retrieval module, a control module and a NAND flash storage module, the interface module, the hardware Embedding module, the vector storage module, the hardware retrieval module, the control module and the NAND flash storage module are coupled and connected through a host internal bus, the interface module adopts a PCIe5.0 interface and is used for high-speed data interaction with a host, receives preprocessed text data blocks and user query texts sent by the host, and feeds back candidate text data or a storage address of the candidate text data obtained through retrieval to the host. The device overcomes the defects of the prior art, such as dependence of RAG system text vectorization and vector retrieval on a host software, low efficiency, high delay and insufficient interface rate of a traditional storage device.

Description

Technical Field

[0001] This invention relates to the field of SSD storage technology and retrieval enhancement generation technology, specifically to an integrated SSD storage device for hardware-accelerated embedding and vector retrieval for retrieval enhancement generation systems. Background Technology

[0002] In Retrieval Enhanced Generation (RAG) systems, text data vectorization and vector retrieval are core components, directly determining the system's response speed and generation quality. The text processing workflow of existing RAG systems typically involves: the host computer preprocessing the original document (parsing, chunking, deduplication, etc.), then using software to call an embedding model to vectorize the text, subsequently storing the generated vector data in a separate vector database. When a retrieval is needed, the host computer again generates a query vector and matches it with the vectors in the vector database, ultimately retrieving the relevant text data and feeding it back to the generation module.

[0003] The aforementioned existing technologies have the following drawbacks: First, both vectorization and vector retrieval rely on host-side software and computing power, consuming a large amount of host CPU / GPU resources, resulting in excessive processing pressure on the host; Second, text data and vector data need to be transmitted multiple times between the host and multiple storage devices (ordinary storage devices, vector databases), resulting in high data transmission latency, especially in large-scale text data processing scenarios, where the response speed decreases significantly; Third, software-implemented vectorization and vector retrieval are inefficient and cannot meet the real-time requirements of RAG systems.

[0004] Existing SSDs only have data storage capabilities and cannot directly perform text vectorization and vector retrieval. Furthermore, some traditional SSDs use SATA interfaces with low transfer rates, which cannot meet the high-efficiency processing requirements of RAG systems. Therefore, there is an urgent need for a hidden SSD that integrates hardware-based vectorization and vector retrieval functions and supports a high-speed PCIe 5.0 interface. This would allow the core processing of the RAG system to be offloaded to the storage hardware, reducing host dependency and data transfer latency, and improving overall processing efficiency. To this end, the inventors have invented a hardware-accelerated embedded SSD storage device for retrieval-enhanced generation systems that integrates both vector retrieval and data processing. Summary of the Invention

[0005] The purpose of this invention is to provide a hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems. This device is rationally designed and overcomes the shortcomings of existing technologies, such as reliance on host software for text vectorization and vector retrieval in RAG systems, low efficiency, high latency, and insufficient interface speed of traditional storage devices. By integrating a dedicated hardware module inside the Phantom SSD and interacting with the host through a high-speed PCIe 5.0 interface, it achieves local vectorization and vector retrieval of preprocessed text data, reduces data transmission links, lowers host resource consumption, and improves the response speed and processing performance of the RAG system, thus solving the problems mentioned in the above-mentioned technical background.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a hardware-accelerated embedding and vector retrieval integrated SSD storage device for retrieval enhancement generation systems. This device includes an interface module, a hardware-enhanced embedding module, a vector storage module, a hardware-enhanced retrieval module, a control module, and a NAND flash memory storage module. These modules are coupled together via a host internal bus to form a closed-loop processing link. The interface module uses a PCIe 5.0 interface for high-speed data interaction with the host, receiving preprocessed text data blocks and user query text from the host, and feeding back the retrieved candidate text data or its storage address to the host. This interface supports the PCIe 5.0 protocol, ensuring... To ensure high-speed and stable data transmission, the hardware embedding module is implemented using a dedicated integrated circuit (ASIC) and is a fixed-function computing circuit. The hardware embedding module includes an input buffer unit, a MAC array, a model parameter storage unit, and an output calibration unit. The input buffer unit temporarily stores preprocessed text data blocks or user query text transmitted by the interface module. The model parameter storage unit stores Sentence-BERT simplified embedded model parameters, which are written via one-time burning or firmware update. The MAC array performs vector space mapping operations on the input text based on the stored embedded model parameters, generating 128-dimensional / 256-dimensional / 512-dimensional text vectors or query vectors. The output calibration unit corrects computational errors, ensuring vector generation accuracy ≥ 99.9%.

[0007] Preferably, the interface module supports the host in transmitting configuration commands to the internal modules of the SSD storage device, including embedded model parameter updates and retrieval threshold settings.

[0008] Preferably, the vector storage module adopts a hybrid storage architecture of high-speed SRAM and high-capacity QLC NAND. The high-speed SRAM is used to store frequently accessed text vectors and vector indexes to improve retrieval speed. The high-capacity QLC NAND is used to store all text vectors and establish a mapping table between the text vectors and the storage addresses of the original preprocessed text data blocks in the NAND flash memory storage module.

[0009] Preferably, the hardware-based retrieval module is implemented using an FPGA, which includes a vector caching unit, a distance calculation unit, a sorting and filtering unit, and a vector indexing unit. The vector caching unit is used to temporarily store the query vector generated by the hardware-based Embedding module and the text vector to be matched retrieved from the vector storage module. The distance calculation unit implements the cosine similarity and Euclidean distance calculation logic in hardware, supports batch vector parallel comparison, and can complete 1024 sets of vector similarity calculations in a single cycle.

[0010] Preferably, the vector indexing unit maintains an HNSW approximate nearest neighbor index structure for quickly locating the text vector to be matched, and the sorting and filtering unit sorts the text vectors according to the similarity calculation results and filters out the top N text vectors with similarity ≥ a preset threshold as candidate text vectors.

[0011] Preferably, the control module adopts a heterogeneous architecture consisting of an FPGA and an MCU. The FPGA is responsible for the timing synchronization of each module, high-speed data forwarding, and hardware algorithm logic scheduling, while the MCU is responsible for running the firmware program, parsing host configuration instructions, monitoring the working status of each module, and coordinating the smooth execution of the write and retrieval processes.

[0012] Preferably, the control module is connected to each module via the host internal bus, and the control module is the core control hub of the SSD storage device.

[0013] Preferably, the NAND flash memory storage module uses QLC NAND flash memory chips with more than 300 layers to store preprocessed raw text data blocks transmitted by the host. It establishes a one-to-one correspondence with the mapping table in the vector storage module to ensure that after the candidate text vector is retrieved, the corresponding raw text data can be quickly located and retrieved.

[0014] Preferably, the device's workflow includes a writing stage and a retrieval stage. The entire process requires no host intervention in vectorization and retrieval operations; the host only needs to complete the initial text preprocessing and the subsequent result reception and generation. High-speed data transmission is achieved through the PCIe 5.0 interface. The writing stage includes the following steps:

[0015] S1: The host preprocesses the original document. The preprocessing includes document parsing, text extraction, deduplication, and semantic segmentation to obtain standardized text data blocks. The text data blocks are sent to the SSD storage device through the PCIe 5.0 interface. The vector dimension can be set through configuration commands.

[0016] S2: After receiving the text data block, the control module transmits it to the input buffer unit of the hardware Embedding module through the host internal bus;

[0017] S3: The hardware Embedding module calls the fixed model parameters in the model parameter storage unit, performs vectorization operations on the text data blocks through the MAC array, generates the corresponding text vectors, and after the output calibration unit corrects the errors of the text vectors, it transmits them to the vector storage module.

[0018] S4: The vector storage module stores text vectors in QLC NAND, caches frequently accessed text vectors in high-speed SRAM, establishes a mapping relationship between text vectors and the storage addresses of the text data blocks in the NAND flash memory storage module, and updates the mapping table.

[0019] S5: The control module sends a write completion signal back to the host via the PCIe 5.0 interface, marking the end of the write phase.

[0020] Preferably, the retrieval stage includes the following steps:

[0021] S1: The host sends the user's query text to the SSD storage device through the PCIe 5.0 interface, and can set the similarity threshold and the number of candidate results N through configuration commands;

[0022] S2: The control module transmits the query text to the hardware embedding module. The hardware embedding module uses the same embedding model as the writing phase to generate the corresponding query vector and transmits it to the vector cache unit of the hardware retrieval module.

[0023] S3: The vector indexing unit of the hardware retrieval module is based on the HNSW index structure. It quickly retrieves the text vector to be matched from the high-speed SRAM or QLC NAND of the vector storage module and transmits it to the vector cache unit.

[0024] S4: The distance calculation unit performs batch parallel similarity calculations on the query vector and the text vector to be matched, and obtains the similarity value between each text vector and the query vector;

[0025] S5: The sorting and filtering unit sorts the text vectors according to the similarity value and filters out the top N text vectors with similarity ≥ preset threshold as candidate text vectors;

[0026] S6: The control module locates the storage address of the original text data block corresponding to the candidate text vector in the NAND flash memory storage module according to the mapping table in the vector storage module, and retrieves the original text data block.

[0027] S7: The control module feeds back the candidate text data block to the host through the PCIe 5.0 interface. The host completes the subsequent search enhancement generation based on the candidate text data block, and the search phase ends.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] 1. This invention provides a hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval-enhanced generation systems. This device includes an interface module, a hardware-based embedding module, a vector storage module, a hardware-based retrieval module, a control module, and a NAND flash memory storage module. The overall structure is simple and rationally designed. The interface module, hardware-based embedding module, vector storage module, hardware-based retrieval module, control module, and NAND flash memory storage module are coupled and connected via an internal host bus to form a closed-loop processing link. The interface module uses a PCIe 5.0 interface, which has the advantage of reducing host resource consumption. The core operations of vectorization and vector retrieval are completed within the Phantom SSD through the hardware module, eliminating the need for host CPU / GPU involvement and significantly reducing host processing pressure.

[0030] 2. This invention can reduce data transmission latency: It adopts a high-speed PCIe 5.0 interface to realize single-channel high-speed interaction between the host and the Phantom SSD. The pre-processed text data and query text only need to be transmitted to the Phantom SSD once. The search results are directly fed back from the Phantom SSD to the host, avoiding the multiple transmissions of text data and vector data between the host and multiple storage devices in the prior art. At the same time, it abandons the SATA interface with a lower transmission rate, reducing data transmission latency by more than 60%.

[0031] 3. This invention can improve processing efficiency: the hardware embedding module and the hardware retrieval module support parallel operation, the vector generation delay is controlled at the microsecond level, and the vector retrieval response time is ≤5ms, which is more than 3 times more efficient than the existing software implementation method. Combined with the high-speed transmission capability of the PCIe 5.0 interface, it further improves the overall data processing throughput.

[0032] 4. The invention has strong adaptability: it supports multiple vector dimension configurations and similarity threshold adjustment, which can adapt to the needs of different RAG systems. It adopts a standard PCIe 5.0 interface, which has good compatibility with existing mainstream host hardware. Moreover, by abandoning the SATA interface, it can focus on high-speed data transmission scenarios, making it easy to promote and apply. Attached Figure Description

[0033] Figure 1 This is a hardware structure block diagram of the present invention;

[0034] Figure 2 This is a flowchart illustrating the writing process of the present invention.

[0035] Figure 3 This is a flowchart illustrating the retrieval phase of the present invention.

[0036] Figure 4 This invention is for the purpose of this invention.

[0037] The reference numerals and names in the figure are as follows:

[0038] 1. Interface module; 2. Hardware Embedding module; 21. Input buffer unit; 22. MAC array; 23. Model parameter storage unit; 24. Output calibration unit; 3. Vector storage module; 31. High-speed SRAM; 32. QLC NAND; 33. Mapping table; 4. Hardware retrieval module; 41. Vector buffer unit; 42. Distance calculation unit; 43. Sorting and filtering unit; 44. Vector indexing unit; 5. Control module; 51. FPGA; 52. MCU; 6. NAND flash memory storage module; 7. Host. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] In the description of the embodiments of the present invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0041] In the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of the present invention according to the specific circumstances.

[0042] Please see Figures 1 to 3This invention provides an embodiment of a hardware-accelerated embedding and vector retrieval integrated SSD storage device for retrieval enhancement generation systems, hereinafter referred to as Phantom SSD. This SSD storage device includes an interface module 1, a hardware-based embedding module 2, a vector storage module 3, a hardware-based retrieval module 4, a control module 5, and a NAND flash memory storage module 6. The interface module 1, hardware-based embedding module 2, vector storage module 3, hardware-based retrieval module 4, control module 5, and NAND flash memory storage module 6 are coupled together via an internal bus of a host 7, forming a closed-loop processing mechanism. The interface module 1 uses a PCIe 5.0 interface and does not support SATA interfaces, which has the advantage of reducing the resource consumption of the host 7. Interface module 1 is used for high-speed data interaction with the host 7. The preprocessed text data and query text only need to be transmitted to the Phantom SSD once. The search results are directly fed back from the Phantom SSD to the host 7, avoiding the multiple transmissions of text data and vector data between the host 7 and multiple storage devices in the existing technology. At the same time, it abandons the low transmission rate of the SATA interface, reducing data transmission latency by more than 60%. It receives the preprocessed text data blocks and user query text sent by the host 7 and feeds back the searched candidate text data or its storage address to the host 7. This interface supports the PCIe 5.0 protocol to ensure high-speed and stable data transmission. Interface module 1 supports the host 7 to transmit configuration instructions to the internal modules of the SSD storage device, including embedded model parameter updates and search threshold settings. The hardware Embedding module 2 is implemented using ASIC. ASIC is a dedicated integrated circuit, which is a fixed-function arithmetic circuit. The hardware Embedding module 2 includes an input cache unit 21, a MAC array 22, a model parameter storage unit 23, and an output calibration unit 24. The MAC array 22 is a multiply-add operation array. The input buffer unit 21 is used to temporarily store the preprocessed text data block or user query text transmitted by the interface module 1. The model parameter storage unit 23 has embedded Sentence-BERT simplified embedded model parameters. The parameters are written by one-time burning or firmware update. The MAC array 22 is used to perform vector space mapping operations on the input text based on the embedded model parameters to generate 128-dimensional / 256-dimensional / 512-dimensional text vectors or query vectors. The output calibration unit 24 is used to correct the calculation error and ensure that the vector generation accuracy is ≥99.9%.

[0043] Specifically, the vector storage module 3 adopts a hybrid storage architecture of high-speed SRAM 31 and large-capacity QLC NAND 32. The high-speed SRAM 31 is used to store frequently accessed text vectors and vector indexes to improve retrieval speed, while the large-capacity QLC NAND 32 is used to store all text vectors and establishes a mapping table 33 between the text vectors and the storage addresses of the original preprocessed text data blocks in the NAND flash memory storage module 6.

[0044] Specifically, the hardware-based retrieval module 4 is implemented using an FPGA 51, which includes a vector caching unit 41, a distance calculation unit 42, a sorting and filtering unit 43, and a vector indexing unit 44. The vector caching unit 41 temporarily stores the query vectors generated by the hardware-based embedding module 2 and the text vectors to be matched retrieved from the vector storage module 3. The distance calculation unit 42 implements the cosine similarity and Euclidean distance calculation logic in hardware, supporting batch vector parallel comparison. It can complete 1024 sets of vector similarity calculations in a single cycle. Therefore, the hardware embedding module 2 and the hardware-based retrieval module 4 support parallel operation. The vector generation latency is controlled at the microsecond level, and the vector retrieval response time is ≤5ms, which is more than 3 times more efficient than the existing software implementation. Combined with the high-speed transmission capability of the PCIe 5.0 interface, the overall data processing throughput is further improved and the processing efficiency is increased. The vector indexing unit 44 maintains the HNSW approximate nearest neighbor index structure. HNSW is a hierarchical navigation small world used to quickly locate the text vector to be matched. The sorting and filtering unit 43 is used to sort the text vectors according to the similarity calculation results and filter out the top N text vectors with similarity ≥ a preset threshold as candidate text vectors. The preset threshold is adjustable between 0.7 and 0.95.

[0045] Specifically, the control module 5 adopts a heterogeneous architecture consisting of FPGA51 and MCU52. FPGA51 is responsible for the timing synchronization of each module, high-speed data forwarding and hardware algorithm logic scheduling. MCU52 is responsible for running firmware programs, parsing configuration instructions of host 7, monitoring the working status of each module, and coordinating the smooth execution of the write and retrieval process. The control module 5 is connected to each module through the internal bus of host 7, and the control module 5 is the core control center of this SSD storage device.

[0046] Specifically, the NAND flash memory storage module 6 uses QLC NAND flash memory chips with more than 300 layers to store the preprocessed raw text data blocks transmitted by the host 7. It establishes a one-to-one correspondence with the mapping table 33 in the vector storage module 3 to ensure that after the candidate text vector is retrieved, the corresponding raw text data can be quickly located and retrieved.

[0047] Working principle: This hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation system operates in a workflow that includes a writing phase and a retrieval phase. The entire process requires no involvement from the host 7 in vectorization and retrieval operations; the host 7 only needs to complete the initial text preprocessing and the subsequent result reception and generation. High-speed data transmission is achieved through the PCIe 5.0 interface. The writing phase includes the following steps:

[0048] S1: Host 7 preprocesses the original document. The preprocessing includes document parsing, text extraction, deduplication, and semantic segmentation. The preprocessing process does not rely on any hardware module of the Phantom SSD and obtains standardized text data blocks. The text data blocks are sent to the SSD storage device through the PCIe 5.0 interface. At the same time, the vector dimension can be set through configuration commands. The vector dimension includes 128-dimensional / 256-dimensional / 512-dimensional.

[0049] S2: After receiving the text data block, the control module 5 transmits it to the input buffer unit 21 of the hardware-based Embedding module 2 through the internal bus of the host 7;

[0050] S3: The hardware Embedding module 2 calls the solidified model parameters in the model parameter storage unit 23, performs vectorization operation on the text data block through the MAC array 22, generates the corresponding text vector, and after the output calibration unit 24 corrects the error of the text vector, it is transmitted to the vector storage module 3.

[0051] S4: Vector storage module 3 stores text vectors in QLC NAND 32, caches frequently accessed text vectors in high-speed SRAM 31, establishes a mapping relationship between text vectors and the storage addresses of the text data blocks in NAND flash memory storage module 6, and updates mapping table 33.

[0052] S5: The control module 5 sends a write completion signal to the host 7 via the PCIe 5.0 interface, and the write phase ends.

[0053] The retrieval phase includes the following steps:

[0054] S1: Host 7 sends the preprocessed user query text to the SSD storage device through the PCIe 5.0 interface, and can set the similarity threshold and the number of candidate results N through configuration commands;

[0055] S2: Control module 5 transmits the query text to hardware embedding module 2. Hardware embedding module 2 uses the same embedding model as the writing stage to generate the corresponding query vector and transmits it to the vector cache unit 41 of hardware retrieval module 4.

[0056] S3: The vector indexing unit 44 of the hardware retrieval module 4 is based on the HNSW index structure. It quickly retrieves the text vector to be matched from the high-speed SRAM 31 or QLC NAND 32 of the vector storage module 3 and transmits it to the vector cache unit 41.

[0057] S4: Distance calculation unit 42 performs batch parallel similarity calculations on the query vector and the text vector to be matched, and obtains the similarity value between each text vector and the query vector;

[0058] S5: The sorting and filtering unit 43 sorts the text vectors according to the similarity value and filters out the top N text vectors with similarity ≥ preset threshold as candidate text vectors.

[0059] S6: The control module 5 locates the storage address of the original text data block corresponding to the candidate text vector in the NAND flash memory storage module 6 according to the mapping table 33 in the vector storage module 3, and retrieves the original text data block.

[0060] S7: Control module 5 feeds back candidate text data blocks to host 7 through PCIe 5.0 interface. Host 7 completes subsequent search enhancement generation based on the candidate text data blocks, and the search phase ends.

[0061] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, including the following process.

[0062] Process 1: Specific selection and connection of hardware modules

[0063] In this embodiment, the specific selection of each hardware module is as follows: Interface module 1 uses the PCIe5 protocol control chip PLX9054, which supports the PCIe5 interface to ensure high-speed data transmission and instruction interaction; Hardware embedding module 2 uses a 28nm process ASIC chip, integrating a 128-channel MAC array 22, which consists of a multiplier AD734 and an accumulator AD8367; Model parameter storage unit 23 uses a 16MB EEPROM to store the Sentence-BERT simplified model parameters; Vector storage module 3 uses a 64K×128bit high-speed static cache chip for its high-speed SRAM 31 and a Kioxia BiCS6 QLC NAND 32. The system uses 300-layer NAND flash memory chips; the hardware retrieval module 4 uses an XC7K325T FPGA51, which integrates a 256-channel parallel comparator array, model LMV339; the control module 5 uses the same XC7K325T FPGA51 as the hardware retrieval module 4, and the MCU52 uses an STM32H743; the NAND flash memory storage module 6 uses a 10TB Kioxia BiCS6 QLC NAND flash memory chip.

[0064] Connection relationships between modules: Interface module 1 connects to host 7 via a PCIe 5 bus and to control module 5 via an internal high-speed bus (AXI 4 bus). Hardware embedding module 2, vector storage module 3, hardware retrieval module 4, and NAND flash memory storage module 6 are all connected to control module 5 via an AXI 4 bus to achieve data and instruction transmission. The output of hardware embedding module 2 is directly connected to the input of vector storage module 3 and hardware retrieval module 4 to achieve direct transmission of vector data, improving efficiency. The entire connection architecture does not include any SATA interface-related connections, ensuring that data transmission relies entirely on the high-speed characteristics of the PCIe 5 interface.

[0065] Step 2: Demonstration of the specific work process

[0066] 2.1 Demonstration of the writing phase

[0067] Host 7 preprocesses a PDF technical document, parses and extracts the text content, and then divides it into standardized text data blocks of 200-1000 tokens according to semantics. The text data blocks are sent to the Phantom SSD of this invention through the PCIe5 interface, and a configuration command is sent to set the vector dimension to 256 dimensions.

[0068] After the MCU52 of the control module 5 parses the instructions, it schedules the timing through the FPGA51 and transmits the text data block to the input buffer unit 21 of the hardware Embedding module 2. The hardware Embedding module 2 calls the Sentence-BERT simplified model parameters embedded in the EEPROM, performs vectorization operations through the 128-channel MAC array 22, generates a 256-dimensional text vector, and after the output calibration unit 24 performs error correction using 7-bit Hamming code, transmits the text vector to the vector storage module 3.

[0069] Vector storage module 3 stores the text vector in QLC NAND 32 and caches the text vector in high-speed SRAM 31. It also records the storage address of the text vector and the corresponding text data block in NAND flash memory storage module 6 in mapping table 33, such as 0x0001-0x0010. After storage is completed, control module 5 sends a write completion signal to host 7 through PCIe 5 interface, and the write phase ends.

[0070] 2.2 Demonstration of the retrieval phase

[0071] The user inputs the query text "core process of RAG system" into the host 7. After preprocessing the query text, the host 7 sends it to the Phantom SSD of this invention through the PCIe5 interface and sends a configuration command to set the similarity threshold to 0.8 and the number of candidate results N=5.

[0072] The control module 5 transmits the query text to the hardware embedding module 2. The hardware embedding module 2 generates a corresponding 256-dimensional query vector based on the 256-dimensional vector configuration and transmits it to the vector cache unit 41 of the hardware retrieval module 4.

[0073] The vector indexing unit 44 of the hardware retrieval module 4 is based on the HNSW index structure. It quickly retrieves 1,000 sets of text vectors to be matched from the high-speed SRAM 31 of the vector storage module 3 and transmits them to the vector cache unit 41. The distance calculation unit 42 performs parallel cosine similarity calculation on the query vector and the 1,000 sets of text vectors to be matched to obtain the similarity value of each text vector. The sorting and filtering unit 43 sorts the similarity values ​​and filters out 5 candidate text vectors with similarity ≥ 0.8.

[0074] According to the mapping table 33 of the vector storage module 3, the control module 5 locates the storage address of the original text data block corresponding to the 5 candidate text vectors in the NAND flash memory storage module 6, retrieves the corresponding text data block, and feeds it back to the host 7 through the PCIe5 interface; the host 7 generates an answer about the "core process of the RAG system" based on the 5 candidate text data blocks and the large model, and the retrieval phase ends.

[0075] Step 3: Performance Test Results

[0076] Performance testing was conducted on the Phantom SSD in this embodiment. The test environment was as follows: Host 7 was configured with an Intel i9-13900K CPU and 64GB of memory; the RAG system used the ChatGPT 3.5 large model; the test data consisted of 100,000 preprocessed text data blocks, with a total capacity of 50GB. The test results are as follows. Figure 4 As shown, the test results indicate that the vectorization speed of text data blocks is 1GB / min, which is 5 times faster than the software implementation; the vector retrieval response time is 3ms, which is 70% faster than the existing "host 7 + ordinary SSD + vector database" solution; the CPU utilization of host 7 is reduced from 60% in the existing solution to 10%; at the same time, relying on the high-speed transmission capability of the PCIe5 interface, the single-channel data transfer rate is more than 8 times faster than the traditional solution using the SATA interface, which significantly improves the overall response efficiency and data throughput of the RAG system.

[0077] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems, characterized in that: The system includes an interface module (1), a hardware embedding module (2), a vector storage module (3), a hardware retrieval module (4), a control module (5), and a NAND flash memory storage module (6). These modules are coupled together via the internal bus of the host (7) to form a closed-loop processing link. The interface module (1) uses a PCIe 5.0 interface for high-speed data interaction with the host (7), receiving preprocessed text data blocks and user query text sent by the host (7), and feeding back the retrieved candidate text data or its storage address to the host (7). The hardware embedding module (2)... Implemented using dedicated integrated circuits, the hardware-based Embedding module (2) is a fixed-function computing circuit. It includes an input buffer unit (21), a MAC array (22), a model parameter storage unit (23), and an output calibration unit (24). The input buffer unit (21) is used to temporarily store the preprocessed text data blocks or user query text transmitted by the interface module (1). The model parameter storage unit (23) has embedded Sentence-BERT simplified version embedded model parameters. The parameters are written by one-time burning or firmware update. The MAC array (22) is used to perform vector space mapping operations on the input text based on the embedded model parameters to generate 128-dimensional / 256-dimensional / 512-dimensional text vectors or query vectors. The output calibration unit (24) is used to correct the calculation error.

2. The hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems according to claim 1, characterized in that: The interface module (1) supports the host (7) in transmitting configuration instructions to the internal modules of the SSD storage device, including embedded model parameter updates and retrieval threshold settings.

3. The hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems according to claim 1, characterized in that: The vector storage module (3) adopts a hybrid storage architecture consisting of high-speed SRAM (31) and QLC NAND (32), wherein the high-speed SRAM (31) is used to store frequently accessed text vectors and vector indexes, and the QLC NAND (32) is used to store all text vectors and establish a mapping table (33) of the storage addresses of text vectors and original preprocessed text data blocks in the NAND flash memory storage module (6).

4. The hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems according to claim 1, characterized in that: The hardware-based retrieval module (4) includes a vector caching unit (41), a distance calculation unit (42), a sorting and filtering unit (43), and a vector indexing unit (44). The vector caching unit (41) is used to temporarily store the query vector generated by the hardware-based Embedding module (2) and the text vector to be matched retrieved from the vector storage module (3). The distance calculation unit (42) implements the cosine similarity and Euclidean distance calculation logic in hardware, supports batch vector parallel comparison, and can complete 1024 sets of vector similarity calculations in a single cycle.

5. The hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems according to claim 4, characterized in that: The vector indexing unit (44) maintains the HNSW approximate nearest neighbor index structure for quickly locating the text vector to be matched. The sorting and filtering unit (43) sorts the text vectors according to the similarity calculation results and filters out the top N text vectors with similarity ≥ preset threshold as candidate text vectors.

6. The hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems according to claim 1, characterized in that: The control module (5) adopts a heterogeneous architecture consisting of FPGA (51) and MCU (52). The FPGA (51) is responsible for the timing synchronization, high-speed data forwarding and hardware algorithm logic scheduling of each module. The MCU (52) is responsible for running the firmware program, parsing the host (7) configuration instructions, monitoring the working status of each module, and coordinating the smooth execution of the writing and retrieval process.

7. The hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems according to claim 6, characterized in that: The control module (5) is connected to each module through the internal bus of the host (7), and the control module (5) is the core control center of the SSD storage device.

8. The hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems according to claim 1, characterized in that: The NAND flash memory storage module (6) uses a QLC NAND flash memory chip with more than 300 layers to store the preprocessed original text data blocks transmitted by the host (7). It establishes a one-to-one correspondence with the mapping table (33) in the vector storage module (3) to ensure that after the candidate text vector is retrieved, the corresponding original text data can be quickly located and retrieved.

9. The hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems according to claim 1, characterized in that, Its workflow includes a writing stage and a retrieval stage. The entire process does not require the host (7) to participate in vectorization and retrieval operations. The host (7) only needs to complete the initial text preprocessing and the subsequent result reception and generation. High-speed data transmission is achieved by relying on the PCIe 5.0 interface. The writing stage includes the following steps: S1: The host (7) preprocesses the original document. The preprocessing includes document parsing, text extraction, deduplication and semantic segmentation to obtain standardized text data blocks. The text data blocks are sent to the SSD storage device through the PCIe 5.0 interface. At the same time, the vector dimension can be set through configuration instructions. S2: After receiving the text data block, the control module (5) transmits it to the input buffer unit (21) of the hardware Embedding module (2) through the internal bus of the host (7). S3: The hardware Embedding module (2) calls the solidified model parameters in the model parameter storage unit (23), performs vectorization operation on the text data block through the MAC array (22), generates the corresponding text vector, and after the output calibration unit (24) performs error correction on the text vector, it is transmitted to the vector storage module (3). S4: The vector storage module (3) stores the text vector in QLC NAND (32), caches the frequently accessed text vector in high-speed SRAM (31), establishes a mapping relationship between the text vector and the storage address of the text data block in the NAND flash memory storage module (6), and updates the mapping table (33). S5: The control module (5) sends a write completion signal to the host (7) through the PCIe 5.0 interface, and the write phase ends.

10. The hardware-accelerated embedded and vector retrieval integrated SSD storage device for retrieval enhancement generation systems according to claim 9, characterized in that, The retrieval phase includes the following steps: S1: The host (7) sends the user query text to the SSD storage device through the PCIe 5.0 interface, and can set the similarity threshold and the number of candidate results N through configuration commands; S2: The control module (5) transmits the query text to the hardware Embedding module (2). The hardware Embedding module (2) uses the same embedding model as the writing stage to generate the corresponding query vector and transmits it to the vector cache unit (41) of the hardware retrieval module (4). S3: The vector index unit (44) of the hardware retrieval module (4) is based on the HNSW index structure and quickly retrieves the text vector to be matched from the high-speed SRAM (31) or QLC NAND (32) of the vector storage module (3) and transmits it to the vector cache unit (41). S4: The distance calculation unit (42) performs batch parallel similarity calculation on the query vector and the text vector to be matched, and obtains the similarity value between each text vector and the query vector; S5: The sorting and filtering unit (43) sorts the text vectors according to the similarity value and filters out the top N text vectors with similarity ≥ preset threshold as candidate text vectors; S6: The control module (5) locates the storage address of the original text data block corresponding to the candidate text vector in the NAND flash memory storage module (6) according to the mapping table (33) in the vector storage module (3) and retrieves the original text data block; S7: The control module (5) feeds back the candidate text data block to the host (7) through the PCIe 5.0 interface. The host (7) completes the subsequent retrieval enhancement generation based on the candidate text data block, and the retrieval stage ends.

Images