An end-side rag implementation method and system based on a fusion architecture

Abstract

The application provides an end-side RAG implementation method and system based on a fusion architecture, the method comprising: initializing a vectorization engine through a user browser to obtain an initialized vector model; performing vectorization processing on a local text knowledge base based on the initialized vector model to obtain a vector knowledge base; obtaining query text issued by a user, and performing retrieval and query on the vector knowledge base based on the query text to output a retrieval result; intelligently deciding on a full-cycle workload based on a current network environment to realize dynamic calculation offloading; and encrypting, storing and verifying full-cycle data to complete the implementation of the end-side RAG. The application replaces a traditional cloud-side vector calculation architecture, realizes zero server cost operation of an AI workflow through a pure front-end calculation architecture, and solves three major pain points of a centralized AI platform, i.e., high cost of computing power, high delay and poor privacy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence application architecture, and particularly relates to an end-side RAG implementation method and system based on a fusion architecture. BACKGROUND

[0002] For the prior art, mainstream AI workflow platforms (Dify / Coze, etc.) adopt a centralized architecture, which has the following defects:

[0003] 1. High vector database authorization fees (such as Pinecone, etc.);

[0004] 2. Exponential growth of GPU server cluster operation and maintenance costs;

[0005] 3. Performance bottlenecks caused by limited user concurrency;

[0006] 4. Compliance risks of data privacy protection;

[0007] For the traditional end-side scheme, the following defects exist:

[0008] 1. Insufficient JavaScript computing performance;

[0009] 2. Lack of hardware acceleration support;

[0010] 3. Difficulty in deploying complex AI models;

[0011] In summary, the prior art lacks an intelligent agent workflow system that does not depend on a central server, and cannot guarantee AI task processing performance while fundamentally optimizing the cost structure of the server. SUMMARY

[0012] To solve the above technical problems, the application provides an end-side RAG implementation method and system based on a fusion architecture, which is used to solve the technical problems in the prior art.

[0013] In one aspect, the application provides the following technical solution, an end-side RAG implementation method based on a fusion architecture, comprising:

[0014] obtaining a user browser, initializing a vectorization engine through the user browser to obtain an initialized vector model;

[0015] obtaining a local text knowledge base of the user, and performing vectorization processing on the local text knowledge base based on the initialized vector model to obtain a vector knowledge base;

[0016] obtaining a query text issued by the user, and performing retrieval and query on the query text and the vector knowledge base to output a retrieval result;

[0017] Identify the full cycle workload, make intelligent decisions based on the current network environment to achieve dynamic calculation unloading, which includes initialization work, vectorization work, retrieval query work;

[0018] Identify the full cycle data, encrypt, store and verify the full cycle data to complete the implementation of the end-side RAG, which includes data in the local text knowledge base, data in the vector knowledge base, and data in the retrieval result.

[0019] Compared with the prior art, the beneficial effects of the present application are: the present application replaces the traditional cloud vector calculation architecture, realizes the zero server cost operation of AI workflow through pure front-end calculation architecture, solves the three major pain points of expensive centralized AI platform computing power, high delay and poor privacy; solve the cost structure problem: eliminate the dependence on expensive vector database, save more than 70% of the centralized server expenditure, avoid the uncontrollable cost generated by the GPU cloud service according to the amount of charge; solve the performance bottleneck problem: break through the HTTP request delay of traditional B / S architecture (reduce the RTT time by an average of 300-500ms), solve the problem of QPS sharp drop caused by insufficient server computing power in high concurrency scenarios; solve the privacy compliance problem: realize the end-to-end closed loop of "from generation to consumption" of user data, avoid uploading sensitive data to the cloud, meet the "data localization" requirement of GDPR and other data sovereignty regulations; solve the architecture flexibility problem: support the running of intelligent agents in offline environment, realize the real edge-cloud hybrid computing elastic deployment.

[0020] Preferably, the step of initializing the vectorization engine through the user browser to obtain an initialized vector model comprises:

[0021] Calling the target interface to detect whether the user browser supports WebGPU API;

[0022] If the user browser supports WebGPU API, request an adapter and a device, obtain the limit computing capability parameters of the device through the limit parameters in the adapter, perform capability negotiation based on the limit computing capability parameters, and load the WASM model optimized and quantized by WebGPU to obtain a target vector engine, and compile the frequently used calculation shader in advance in the target vector engine to obtain an initialized vector model.

[0023] If the user browser does not support WebGPU API, a degradation strategy will be executed, and the model will be loaded and run based on the degradation strategy using WebGL and TensorFlow.js to obtain an initialized vector model.

[0024] Preferably, the step of vectorizing the local text knowledge base based on the initialization vector model to obtain a vector knowledge base includes:

[0025] The long text data in the local text knowledge base is divided into blocks according to a preset length to obtain several text blocks;

[0026] The text blocks are input into the initialization vector model, and each text block is processed in parallel by a GPU computing unit to generate a high-dimensional vector representation. The dimension of the high-dimensional vector representation is dynamically selected based on the limit computing capability parameter. To obtain the generated vector:

[0027] ;

[0028] In the formula, For user equipment value, for Threshold;

[0029] The generated vector is compressed into an integer vector using the preset instruction set provided by the WASM model;

[0030] The integer vectors are stored in linear memory within the WASM model, and an efficient nearest neighbor search index is constructed using a hierarchical navigation small world graph to obtain the vector knowledge base.

[0031] Preferably, the step of performing a retrieval query based on the query text and the vector knowledge base to output retrieval results includes:

[0032] The query text is sent to a Web Worker thread, where the query text is vectorized by initializing a vector model to obtain a query text vector.

[0033] Based on the hierarchical navigation small-world graph, the candidate vector most similar to the query text vector is searched in the vector knowledge base;

[0034] Calculate the candidate vector With the query text vector Similarity between :

[0035] ;

[0036] In the formula, Representing candidate vectors respectively Query text vector The first in Word vectors, The number of word vectors. To prevent zero value;

[0037] Select the maximum similarity of the first few candidate vectors as the search results, and attach the original text position pointer in the search results.

[0038] Preferably, the step of making intelligent decisions on the full-cycle workload based on the current network environment to realize dynamic calculation offloading comprises:

[0039] Measuring the current available bandwidth of the network used by the user Based on the current available bandwidth Calculate the offload value :

[0040] ;

[0041] In the formula, The task calculation of the full-cycle workload, The local calculation time threshold;

[0042] If the offload value is , all the workload in the full-cycle workload is forced to be completed locally;

[0043] If the offload value is , the most intensive task workload in the full-cycle workload is offloaded to the cloud, and the remaining work in the full-cycle workload is completed locally to realize dynamic calculation offloading.

[0044] Preferably, the step of identifying the full-cycle data, encrypting, storing and verifying the full-cycle data to complete the implementation of the end-side RAG comprises:

[0045] Use the WebCrypto API built-in user browser and adopt the preset encryption algorithm to encrypt the full-cycle data to obtain encrypted data;

[0046] Store the encrypted data in the IndexedDB of the user browser, generate an encryption key by the user password through the key derivation function, and send the encryption key to the user;

[0047] Identify the sensitive operation of the user, and verify the sensitive operation through the WASM model to complete the implementation of the end-side RAG:

[0048] ;

[0049] In the formula, The verification function, Public parameters, input data, and proof, respectively.

[0050] In a second aspect, the present application provides the following technical solutions: a terminal-side RAG implementation system based on a fusion architecture, the system comprising:

[0051] An initialization module is configured to acquire a user browser, initialize a vectorization engine through the user browser, and obtain an initialized vector model.

[0052] A vectorization module is configured to acquire a local text knowledge base of a user, perform vectorization processing on the local text knowledge base based on the initialized vector model, and obtain a vector knowledge base.

[0053] A query module is configured to acquire a query text issued by a user, perform retrieval and query based on the query text and the vector knowledge base, and output a retrieval result.

[0054] An unloading module is configured to identify a full-cycle workload, intelligently determine the full-cycle workload based on a current network environment, and perform dynamic calculation and unloading, wherein the full-cycle workload comprises initialization work, vectorization work, and retrieval and query work.

[0055] A verification module is configured to identify full-cycle data, encrypt, store, and verify the full-cycle data, and complete the implementation of the terminal-side RAG, wherein the full-cycle data comprises data in the local text knowledge base, data in the vector knowledge base, and data in the retrieval result.

[0056] In a third aspect, the present application provides the following technical solutions: a computer comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the terminal-side RAG implementation method based on the fusion architecture when executing the computer program.

[0057] In a fourth aspect, the present application provides the following technical solutions: a storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the terminal-side RAG implementation method based on the fusion architecture. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0059] Figure 1 A flowchart of the terminal-side RAG implementation method based on the fusion architecture provided by the first embodiment of the present application;

[0060] Figure 2A structural block diagram of an end-side RAG implementation system based on a fusion architecture is provided for Embodiment Two of the present application.

[0061] Figure 3 A hardware structure schematic diagram of a computer is provided for another embodiment of the present application.

[0062] Embodiments of the present application will be further described below with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the embodiments of the present application, and cannot be understood as a limitation of the present application.

[0064] Embodiment One

[0065] In Embodiment One of the present application, as shown in Figure 1 a fusion architecture-based end-side RAG implementation method, comprising:

[0066] S1, obtaining a user browser, initializing a vectorization engine through the user browser to obtain an initialized vector model;

[0067] Specifically, the user browser here specifically refers to the browser of the user end, and this step is specifically used to establish a high-performance heterogeneous computing environment in the user browser, to prepare for subsequent vectorization calculation.

[0068] The step S1 comprises:

[0069] S11, calling a target interface to detect whether the user browser supports WebGPU API;

[0070] Specifically, the target interface here is navigator.gpu interface, and by detecting whether WebGPU API is supported, it is the premise of enabling high-performance GPU accelerated calculation.

[0071] S12, if the user browser supports WebGPU API, requesting an adapter and a device, obtaining a limit computing capability parameter of the device through a limit parameter in the adapter, performing capability negotiation based on the limit computing capability parameter and loading a WebGPU optimized and quantized WASM model to obtain a target vector engine, and compiling a frequently used computing shader in advance in the target vector engine to obtain an initialized vector model;

[0072] Specifically, the limit parameter here is adapter.limits, which is a set of actual limit parameters for checking the physical GPU, and the limit computing capability parameters here include the maximum number of computing units, buffer size, etc., and after the WebGPU optimization and quantization of the WASM model, the output model file is an FP16 precision model file, to achieve faster loading and higher computing efficiency;

[0073] At the same time, the frequently used computing shader is compiled in advance in the target vector engine, and the computing shader here can be a matrix multiplication (MMUL) kernel function, and the process is a warm-up optimization process, avoiding the delay introduced by real-time compilation when performing key calculations for the first time.

[0074] S13, if the user browser does not support WebGPU API, a downgrade strategy will be executed, and based on the downgrade strategy, the model is loaded and run using WebGL and TensorFlow.js to get the initialized vector model;

[0075] Specifically, if it is not supported, an automatic downgrade strategy is executed to ensure the compatibility and basic functionality of the system. WebGL (Web Graphics Library) is an API for rendering 3D graphics in web browsers. It allows developers to use JavaScript and a subset of OpenGL ES 2.0 to create complex graphics and animations. The main advantage of WebGL is that it can render 3D graphics directly in the browser without the need to install any plugins. TensorFlow.js is a JavaScript-based machine learning library developed by Google, which allows developers to run machine learning models in the browser. TensorFlow.js supports running TensorFlow models in the browser and can take advantage of WebGL for GPU acceleration. In addition, TensorFlow.js also provides many high-level APIs, making it easier for developers to create and train machine learning models.

[0076] S2, obtain the local text knowledge base of the user, and perform vectorization processing on the local text knowledge base based on the initialized vector model to obtain a vector knowledge base;

[0077] Specifically, this step is to convert the user's local text knowledge base into an efficient and quickly searchable vector index, all of which are completed on the browser side.

[0078] Wherein, the step S2 comprises:

[0079] S21, block the long text data in the local text knowledge base according to a preset length to obtain a plurality of text blocks.

[0080] Specifically, the preset length here is 64 characters, which is to adapt to the parallel computing architecture of the GPU and split the large task into numerous small task units that can be processed in parallel.

[0081] S22, input the text block into the initialization vector model, each text block is processed by one GPU computing unit in parallel to generate a high-dimensional vector representation, and the dimension of the high-dimensional vector representation is dynamically selected according to the limit computing power parameter to obtain the generated vector:

[0082] ;

[0083] wherein, is the value of the user equipment, is a threshold value;

[0084] wherein, the value here is a unit for measuring computer performance, representing the number of floating point operations per second.

[0085] S23, compress the generated vector into an integer vector using the preset instruction set provided by the WASM model;

[0086] Specifically, the generated vector (FP32 / FP16) can be compressed into an integer vector (INT8) by compression, and the preset instruction set here is the SIMD (Single Instruction Multiple Data) instruction set.

[0087] S24, store the integer vector in the linear memory in the WASM model, and use the hierarchical navigation small world graph to build an efficient neighbor search index to obtain a vector knowledge base;

[0088] Specifically, the hierarchical navigation small world graph is an index structure that greatly reduces the time complexity of retrieval from O(n) to O(log n), laying the foundation for real-time retrieval.

[0089] S3, obtain the query text issued by the user, and perform retrieval query based on the query text and the vector knowledge base to output the retrieval result;

[0090] Specifically, the purpose of this step is to respond to the user's real-time query and quickly find the most relevant information in the local knowledge base.

[0091] Wherein, the step S3 comprises:

[0092] S31, send the query text to a Web Worker thread, and vectorize the query text by initializing a vector model in the Web Worker thread to obtain a query text vector;

[0093] Specifically, the Web Worker thread can avoid blocking the user interface.

[0094] S32, search for candidate vectors most similar to the query text vector in the vector knowledge base based on the hierarchical navigation small world graph.

[0095] S33, calculate the similarity between the candidate vectors and the query text vector .

[0096] ;

[0097] In the formula, respectively represent the first word vector in the candidate vector , the query text vector , the number of word vectors is , and the zero value is prevented.

[0098] Specifically, the zero value here is , and the purpose is to prevent zero vectors from causing division by zero errors. The numerator uses SIMD parallel computing dot product, which is 6-8 times faster than JavaScript in WASM. Practice shows that this improvement improves the retrieval quality of low-dimensional vectors (384 dimensions) by 12%.

[0099] S34, select the top several candidate vectors with the highest similarity as the retrieval result, and attach the original text position pointer in the retrieval result;

[0100] Specifically, in the present application, the top 3 results are selected as the final retrieval result. Finally, the Web Worker returns the top 3 retrieval results with the highest similarity and their position information in the original knowledge base to the main thread, and the main thread obtains and displays the corresponding original text segment accordingly.

[0101] S4, identify the full-cycle workload, and intelligently decide the full-cycle workload based on the current network environment to realize dynamic calculation offloading, wherein the full-cycle workload includes initialization work, vectorization work, and retrieval query work.

[0102] Specifically, the purpose of this step is to make the system have the flexibility of end-cloud collaboration, and intelligently distribute the calculation tasks according to the current network and device state. ​​

[0103] wherein the step S4 comprises:

[0104] S41, measuring the current available bandwidth of the network used by the user , based on the current available bandwidth calculating an offloading value :

[0105] ;

[0106] wherein, is the task amount of the full-cycle workload, is a local computing time threshold value;

[0107] S42, if the offloading value is , then all of the full-cycle workload is completed locally;

[0108] S43, if the offloading value is , then the most intensive task work in the full-cycle workload is offloaded to the cloud and the remaining work in the full-cycle workload is completed locally to achieve dynamic computing offloading;

[0109] Specifically, in the present application, the most intensive work is the similarity comparison of high-dimensional vectors, i.e., the above-mentioned process is a cloud collaboration mode, but it does not upload all data, but only offloads the most intensive task (such as similarity comparison of high-dimensional vectors) to the cloud, and the remaining work is still completed locally, so as to reduce the data transmission amount and balance the load.

[0110] S5, identifying full-cycle data, encrypting, storing and verifying the full-cycle data to complete the implementation of the end-side RAG, wherein the full-cycle data includes data in the local text knowledge base, data in the vector knowledge base, and data in the search results;

[0111] Specifically, this step runs through the entire life cycle of data processing, ensuring the confidentiality and privacy security of user data.

[0112] wherein the step S5 comprises:

[0113] S51, using the WebCrypto API built-in in the user's browser and using a preset encryption algorithm to encrypt the full-cycle data to obtain encrypted data;

[0114] Specifically, the preset encryption algorithm here is the AES-GCM algorithm.

[0115] S52, storing the encrypted data in the IndexedDB of the user's browser, generating an encryption key by a key derivation function through the user's password and sending the encryption key to the user;

[0116] Specifically, the IndexedDB is used for persistent storage, and the encryption key is generated by a user password through a key derivation function (such as PBKDF2), so that the key is not separated from the end, and the data is stored in ciphertext.

[0117] S53, sensitive operations of a user are identified, and the sensitive operations are verified through a WASM model to complete implementation of end-side RAG:

[0118] ;

[0119] In the formula, is a verification function, are respectively a public parameter, input data, and proof;

[0120] Specifically, the sensitive operation can be that the knowledge base has been retrieved but the content thereof is not leaked, and zero-knowledge proof (such as zk-SNARKs) is used for verification, and the verification logic is implemented by efficient WASM, and the verifier does not need to contact any original data to verify the authenticity of the calculation process, so that the risk of data leakage is fundamentally eliminated.

[0121] The above steps are connected with each other, and together constitute a complete, efficient, safe and extremely low-cost end-side intelligent agent RAG solution.

[0122] The end-side RAG implementation method based on the fusion architecture provided by the embodiment one of the application replaces the traditional cloud-side vector calculation architecture, realizes zero-service-end cost operation of the AI workflow through a pure front-end calculation architecture, solves three pain points of high cost, high delay and poor privacy of the centralized AI platform, solves the cost structure problem: eliminates the dependence on expensive vector databases, saves more than 70% of the centralized server expenses, and avoids uncontrollable costs generated by the GPU cloud service according to the amount; solve the performance bottleneck problem: break through the HTTP request delay of the traditional B / S architecture (reduce the RTT time by 300-500 ms on average), solve the problem of sharp reduction of QPS caused by insufficient server computing power in a high-concurrency scenario; solve the privacy compliance problem: realize end-to-end closed loop of "from generation to consumption" of user data, avoid uploading sensitive data to the cloud, meet the "data localization" requirement of the data sovereignty regulations such as GDPR; solve the architecture flexibility problem: support intelligent agent running in an offline environment, and realize real edge-cloud hybrid computing elastic deployment.

[0123] Embodiment two

[0124] As shown in Figure 2 The embodiment two of the application provides a kind of end-side RAG implementation system based on fusion architecture, the system includes:

[0125] An initialization module 1 is configured to acquire a user browser, and initialize a vectorization engine through the user browser to obtain an initialized vector model;

[0126] A vectorization module 2 is configured to acquire a local text knowledge base of a user, and perform vectorization processing on the local text knowledge base based on the initialized vector model to obtain a vector knowledge base;

[0127] A query module 3 is configured to acquire a query text issued by a user, and perform retrieval query on the query text and the vector knowledge base to output a retrieval result;

[0128] An unloading module 4 is configured to identify a full-cycle workload, and perform intelligent decision on the full-cycle workload based on a current network environment to realize dynamic calculation unloading, wherein the full-cycle workload includes initialization work, vectorization work and retrieval query work.

[0129] A verification module 5 is configured to identify full-cycle data, and perform encryption, storage and verification on the full-cycle data to complete the realization of an end-side RAG, wherein the full-cycle data includes data in the local text knowledge base, data in the vector knowledge base and data in the retrieval result.

[0130] The initialization module 1 includes:

[0131] A calling sub-module is configured to call a target interface to detect whether the user browser supports a WebGPU API;

[0132] A first initialization sub-module is configured to, if the user browser supports the WebGPU API, request an adapter and a device, acquire a limit calculation capability parameter of the device through a limit parameter in the adapter, perform capability negotiation based on the limit calculation capability parameter, and load a WASM model optimized and quantized by WebGPU to obtain a target vector engine, pre-compile a frequently used calculation shader in the target vector engine to obtain an initialized vector model;

[0133] A second initialization sub-module is configured to, if the user browser does not support the WebGPU API, execute a degradation strategy, and based on the degradation strategy, push back to use WebGL and TensorFlow.js to load and run a model to obtain an initialized vector model.

[0134] The vectorization module includes:

[0135] A blocking sub-module is configured to block long text data in the local text knowledge base according to a preset length to obtain a plurality of text blocks;

[0136] The generating submodule is configured to input the text blocks into the initialization vector model, each text block is processed by one GPU computing unit in parallel, generate a high-dimensional vector representation, and dynamically select the dimension of the high-dimensional vector representation according to a limit computing capacity parameter to obtain a generated vector:

[0137] ;

[0138] wherein, is a value of the user equipment ; is a threshold value ;

[0139] The compressing submodule is configured to compress the generated vector into an integer vector using a preset instruction set provided by the WASM model.

[0140] The indexing submodule is configured to store the integer vector in linear memory in the WASM model, and construct an efficient neighbor search index using a hierarchical navigation small world graph, to obtain a vector knowledge base.

[0141] The query module 3 comprises:

[0142] The vectorizing submodule is configured to send the query text to a Web Worker thread, and perform vectorization processing on the query text in the Web Worker thread by using the initialization vector model, to obtain a query text vector.

[0143] The searching submodule is configured to search for candidate vectors most similar to the query text vector in the vector knowledge base based on the hierarchical navigation small world graph.

[0144] The calculating submodule is configured to calculate the similarity between the candidate vectors and the query text vector :

[0145] ;

[0146] wherein, denote the first word vectors in the candidate vectors , the query text vector , respectively, is the number of word vectors, is a zero value prevention;

[0147] The retrieval output submodule is configured to select the first several candidate vectors with the largest similarity as retrieval results, and attach original text position pointers in the retrieval results.

[0148] ​The offloading module 4 comprises:

[0149] an offloading value submodule for measuring the current available bandwidth of the network used by the user , calculating an offloading value based on the current available bandwidth :

[0150] ;

[0151] wherein, is the task amount of the full-cycle workload, is a local computing time threshold value;

[0152] a local submodule for, if the offloading value is , forcing all workloads in the full-cycle workload to be completed locally;

[0153] a cooperative submodule for, if the offloading value is , offloading the most intensive task work in the full-cycle workload to the cloud and completing the remaining work in the full-cycle workload locally to achieve dynamic computing offloading.

[0154] The verification module 5 comprises:

[0155] an encryption submodule for performing encryption processing on the full-cycle data using the WebCrypto API built in the user browser and adopting a preset encryption algorithm to obtain encrypted data;

[0156] a storage submodule for storing the encrypted data in the IndexedDB of the user browser, generating an encryption key by a key derivation function through the user password and sending the encryption key to the user;

[0157] a verification submodule for identifying sensitive operations of the user and verifying the sensitive operations through a WASM model to complete the implementation of the end-side RAG:

[0158] ;

[0159] wherein, is a verification function, are respectively a public parameter, input data and proof.

[0160] In some other embodiments of the present application, the present application provides the following technical solutions, a computer comprising a memory 102, a processor 101 and a computer program stored on the memory 102 and capable of running on the processor 101, wherein the processor 101 implements the end-side RAG implementation method based on the fusion architecture as described above when executing the computer program. ​

[0161] In particular, the processor 101 can include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present application.

[0162] The memory 102 can include mass storage for data or instructions. By way of example, and not limitation, the memory 102 can include a Hard Disk Drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash drive, a Compact Disk Drive (CD Drive), a DVD Drive, a tape drive, a USB drive, or any combination of two or more of these. The memory 102 can be removable and / or non-removable (or fixed) as appropriate. The memory 102 can be internal or external as appropriate. In certain embodiments, the memory 102 is a non-volatile memory. In certain embodiments, the memory 102 includes a Read-Only Memory (ROM) and a Random Access Memory (RAM). The ROM can be a mask-programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), an Electrically Alterable ROM (EAROM), or a FLASH memory, or any combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended Data Output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0163] The memory 102 can be used to store or buffer various data files needed for processing and / or communication, and possible computer program instructions executed by the processor 101.

[0164] The processor 101 implements the above-mentioned end-side RAG implementation method based on the fusion architecture by reading and executing the computer program instructions stored in the memory 102.

[0165] In some embodiments, the computer can further include a communication interface 103 and a bus 100. As shown, the processor 101, the memory 102, and the communication interface 103 are connected through the bus 100 and complete communication with each other. Figure 3

[0166] The communication interface 103 is used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application. The communication interface 103 can also realize data communication with other components, such as external devices, image / data acquisition devices, databases, external storage, image / data processing workstations, etc.

[0167] ​Bus 100 includes hardware, software, or both, to couple components of computer device to each other and to couple components of computer device to other devices. Bus 100 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, a local bus, etc. By way of example and not limitation, bus 100 can include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 100 can include one or more buses. Although the present embodiments describe and show a particular bus, the present embodiments contemplate any suitable bus or interconnect.

[0168] The computer can be based on the acquisition of the end-side RAG implementation system based on the fusion architecture, and execute the end-side RAG implementation method based on the fusion architecture of the present application, so as to realize the end-side RAG implementation based on the fusion architecture.

[0169] In some embodiments of the present application, in combination with the above-mentioned end-side RAG implementation method based on the fusion architecture, the present embodiments provide the following technical solutions: a storage medium, the storage medium has a computer program stored thereon, and the computer program is executed by a processor to realize the above-mentioned end-side RAG implementation method based on the fusion architecture.

[0170] Those skilled in the art will appreciate that the logic and / or steps represented in the flow diagrams, or otherwise described herein, for example, can be thought of as a list of executable instructions for implementing logic functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination thereof. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical), and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via the optical scanner of a device or device, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0171] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical), and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via the optical scanner of a device or device, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0172] It should be understood that aspects of the present application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or combinations thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and so forth.

[0173] The technical features of the above-described embodiments can be combined in any manner, and in order to make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered within the scope of the present specification.

[0174] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the protection scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.

Claims

1. An end-side RAG implementation method based on a fusion architecture, characterized in that, The method comprises the following steps: obtaining a user browser, initializing a vectorization engine through the user browser to obtain an initialized vector model; obtaining a local text knowledge base of the user, and performing vectorization processing on the local text knowledge base based on the initialized vector model to obtain a vector knowledge base; obtaining a query text issued by the user, and performing retrieval query based on the query text and the vector knowledge base to output a retrieval result; identifying a full-cycle workload, and intelligently deciding the full-cycle workload based on a current network environment to realize dynamic calculation offloading, wherein the full-cycle workload includes initialization work, vectorization work and retrieval query work; identifying full-cycle data, and performing encryption, storage and verification on the full-cycle data to complete the realization of end-side RAG, wherein the full-cycle data includes data in the local text knowledge base, data in the vector knowledge base and data in the retrieval result; the step of intelligently deciding the full-cycle workload based on the current network environment to realize dynamic calculation offloading comprises: Measuring the current available bandwidth of a network used by a user , based on the current available bandwidth calculating an offload value : ； In the formula, the amount of tasks for the full-cycle workload, is a local computation time threshold; if the offload value is then all of the work in the full cycle workload is forced to be completed locally; if the offloading value is then offloading the most intensive task work in the full-cycle workload to the cloud and completing the rest of the work in the full-cycle workload locally to achieve dynamic computation offloading; the step of identifying full-cycle data and performing encryption, storage and verification on the full-cycle data to complete the realization of end-side RAG comprises: using the WebCrypto API built in the user browser and using a preset encryption algorithm to perform encryption processing on the full-cycle data to obtain encrypted data; storing the encrypted data in the IndexedDB of the user browser, generating an encryption key by a key derivation function through a user password and sending the encryption key to the user; identifying sensitive operations of the user and verifying the sensitive operations through a WASM model to complete the realization of end-side RAG: ； In the formula, To verify the function, Public parameter, input data, and proof, respectively.

2. The end-side RAG implementation method based on the fusion architecture according to claim 1, characterized in that, the step of initializing the vectorization engine through the user browser to obtain the initialized vector model comprises: calling a target interface to detect whether the user browser supports WebGPU API; if the user browser supports WebGPU API, requesting an adapter and a device, obtaining limit computing capability parameters of the device through limit parameters in the adapter, performing capability negotiation based on the limit computing capability parameters and loading a WASM model optimized and quantized by WebGPU to obtain a target vector engine, compiling a high-frequency used computing shader in advance in the target vector engine to obtain the initialized vector model; if the user browser does not support WebGPU API, executing a degradation strategy, and based on the degradation strategy, returning to use WebGL and TensorFlow.js to load and run a model to obtain the initialized vector model.

3. The end-side RAG implementation method based on the fusion architecture according to claim 1, characterized in that, the step of performing vectorization processing on the local text knowledge base based on the initialized vector model to obtain the vector knowledge base comprises: blocking long text data in the local text knowledge base according to a preset length to obtain a plurality of text blocks; inputting the text blocks into the initialization vector model, each text block being processed in parallel by one GPU computing unit, generating high-dimensional vector representations, and dynamically selecting dimensions of the high-dimensional vector representations according to a limit computing power parameter to obtain a generation vector: ； In the formula, a user equipment value, is threshold value; using a preset instruction set provided by the WASM model to compress the generated vector into an integer vector; storing the integer vector in linear memory in the WASM model, and using a hierarchical navigation small world graph to construct an efficient neighbor search index to obtain the vector knowledge base.

4. The end-side RAG implementation method based on the fusion architecture according to claim 1, characterized in that, The step of performing a retrieval query based on the query text and the vector knowledge base to output a retrieval result comprises: sending the query text to a Web Worker thread, and performing vectorization processing on the query text by initializing a vector model in the Web Worker thread to obtain a query text vector; searching for candidate vectors most similar to the query text vector in the vector knowledge base based on a hierarchical navigation small world graph; computing the candidate vector similarity between the query text vector :​ ； wherein, represents the candidate vector , the query text vector , the i-th word vector in the candidate vector , the i-th word vector in the query text vector , the number of word vectors, and is a zero value prevention. selecting the first several candidate vectors with the largest similarity as the retrieval result, and attaching an original text location pointer to the retrieval result.

5. A system for implementing end-side RAG based on a fusion architecture, which adopts the method for implementing end-side RAG based on a fusion architecture according to claim 1, characterized in that, The system comprises: an initialization module configured to obtain a user browser, and initialize a vectorization engine through the user browser to obtain an initialized vector model; a vectorization module configured to obtain a local text knowledge base of a user, and perform vectorization processing on the local text knowledge base based on the initialized vector model to obtain a vector knowledge base; a query module configured to obtain a query text issued by a user, and perform a retrieval query based on the query text and the vector knowledge base to output a retrieval result; an unloading module configured to identify a full-cycle workload, and intelligently decide the full-cycle workload based on a current network environment to realize dynamic calculation unloading, wherein the full-cycle workload comprises initialization work, vectorization work, and retrieval query work; a verification module configured to identify full-cycle data, and perform encryption, storage, and verification on the full-cycle data to complete the realization of an end-side RAG, wherein the full-cycle data comprises data in the local text knowledge base, data in the vector knowledge base, and data in the retrieval result.

6. A computer comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the end-side RAG realization method based on the fusion architecture according to any one of claims 1 to 4.

7. A storage medium, characterized by The storage medium has the computer program stored thereon, and the computer program is executed by the processor to realize the end-side RAG realization method based on the fusion architecture according to any one of claims 1 to 4.

Images