Vector retrieval system for browser environment

Abstract

This application provides a vector retrieval system for browser environments, relating to the field of vector retrieval technology, aiming to achieve efficient vector retrieval in browser environments. The system includes a task scheduling module and a hierarchical data management module. The task scheduling module receives user query vector data and schedules the hierarchical data management module to execute vector retrieval tasks. The hierarchical data management module includes a computation layer storing a first part of frequently accessed vector data, an intermediary layer storing a second part of frequently accessed vector data, and a storage layer storing all vector data. In response to a vector retrieval task, the computation layer retrieves target vector data and query vector data from its own memory to perform vector retrieval calculations. Furthermore, when target vector data is missing from its own memory, the computation layer retrieves the missing target vector data and query vector data in batches from the intermediary layer and / or from the storage layer through the intermediary layer to perform vector retrieval calculations.

Description

Technical Field

[0001] This application relates to the field of vector retrieval technology, and in particular to a vector retrieval system for browser environments. Background Technology

[0002] With the rapid development of artificial intelligence technology, vector retrieval technology plays a crucial role in areas such as retrieval enhancement, recommendation systems, and semantic search. Especially in privacy-sensitive scenarios, browser-based vector retrieval engines, capable of processing user data locally without uploading to the cloud, have become an important technological means for protecting user privacy in fields such as finance, healthcare, and education. However, as a cross-platform, lightweight operating environment, browsers face significant challenges from the resource-intensive nature of vector retrieval technology, which posed substantial challenges to browser computing power, memory management, and storage access mechanisms.

[0003] Among the related technologies, the Hierarchical Navigable SmallWorld (HNSW) algorithm is mainly used as the index architecture for the browser-side vector retrieval engine. This index architecture extends the storage space to the device disk through IndexedDB (an index-based database) to solve the browser's memory limitation problem. Furthermore, a prefetching mechanism is used to reduce the frequency of IndexedDB storage access, thereby reducing the overall IndexedDB storage access latency and improving the speed of vector retrieval.

[0004] While these solutions have initially achieved vector retrieval functionality in browser environments, they still face at least the following technical bottlenecks in practical deployments: First, there's the computational performance bottleneck. The parsing and execution characteristics of the browser's JavaScript engine (a lightweight, interpreted programming language) lead to inefficient computationally intensive operations, making it difficult to meet the real-time interactive needs of browsers. Second, there's the storage access efficiency bottleneck. The heuristic data loading mechanism using a fixed prefetch size loads missing data from IndexedDB storage into memory, resulting in significant redundant data loading issues. Therefore, how to achieve efficient vector retrieval in browser environments is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] In view of the above problems, embodiments of this application provide a vector retrieval system for browser environments in order to overcome or at least partially solve the above problems.

[0006] This application provides a vector retrieval system for browser environments, including a task scheduling module and a hierarchical data management module, wherein:

[0007] The task scheduling module receives the user's query vector data and schedules the hierarchical data management module to execute the vector retrieval task. The query vector data represents the query information provided by the user.

[0008] The hierarchical data management module includes a computation layer storing a first part of high-frequency access vector data, an intermediary layer storing a second part of high-frequency access vector data, and a storage layer storing all vector data. The high-frequency access vector data represents vector data that has been accessed more than a threshold number of times within a target time period.

[0009] In response to the vector retrieval task, the hierarchical data management module performs vector retrieval calculations by retrieving target vector data and query vector data from its own memory, and by retrieving missing target vector data and query vector data in batches from the intermediary layer and / or from the storage layer through the intermediary layer when target vector data is missing from its own memory.

[0010] Optionally, the computation layer is implemented based on WebAssembly, and the computation layer includes a hierarchical index structure and a caching module. Each layer of the hierarchical index structure consists of identifiers for vector data.

[0011] The computation layer retrieves the target vector data and the query vector data from its own memory to perform vector retrieval calculations, including:

[0012] The cache module of the computing layer, according to the hierarchical index structure, sequentially retrieves the target vector data of the current layer from its own memory based on the identifier of the current layer, and performs vector retrieval calculation of the current layer with the query vector data.

[0013] Optionally, the intermediary layer includes a first application programming interface; when the computation layer lacks target vector data in its own memory, it batch retrieves the missing target vector data from the intermediary layer and / or from the storage layer through the intermediary layer, along with the query vector data, to perform vector retrieval computation, including:

[0014] When the computing layer is missing the target vector data of the current layer in its own memory, it notifies the intermediary layer to check the heap cache through the first application programming interface of the intermediary layer, retrieves the missing target vector data of the current layer from the heap cache of the intermediary layer, and performs vector retrieval calculation with the query vector data;

[0015] If the target vector data of the current layer is not missing in the heap cache of the intermediate layer, the missing target vector data of the current layer is obtained from the storage layer through the intermediate layer and vector retrieval calculation is performed with the query vector data.

[0016] Optionally, the intermediary layer further includes a second application programming interface and a third application programming interface;

[0017] The computation layer updates the vector data in its own memory to the intermediary layer and the storage layer for storage through the second application programming interface of the intermediary layer;

[0018] The computation layer suspends the target task to the end of the event queue through the third application programming interface of the intermediary layer, waiting for the intermediary layer to read the missing target vector data of the current layer from the storage layer. After the intermediary layer completes the reading of the missing target vector data of the current layer from the storage layer, the computation layer continues to execute the target task. The target task represents obtaining the target vector data of the current layer and performing vector retrieval calculation of the current layer with the query vector data.

[0019] Optionally, when the computation layer lacks target vector data in its own memory, it batch retrieves the missing target vector data from the intermediary layer and / or from the storage layer through the intermediary layer, along with the query vector data, to perform vector retrieval calculations, including:

[0020] When the computing layer lacks target vector data in its own memory and meets the data loading conditions, it batch retrieves the missing target vector data and the query vector data from the intermediary layer and / or from the storage layer through the intermediary layer to perform vector retrieval calculations. The data loading conditions are used to trigger the batch loading of the missing target vector data in the computing layer's own memory.

[0021] Optionally, the storage layer includes a data access module, which manages the data loading conditions, including a first data loading condition and a second data loading condition.

[0022] The first data loading condition is as follows: during the process of retrieving the target vector data of the current layer from its own memory according to the identifier of the current layer and performing the current layer vector retrieval calculation with the query vector data, if the number of missing target vector data of the current layer is less than or equal to the missing threshold, the loading of the missing target vector data corresponding to the current layer is temporarily suspended, and after retrieving the target vector data of the current layer from its own memory according to the identifier of the current layer and performing the current layer vector retrieval calculation with the query vector data, batch loading is triggered;

[0023] The second data loading condition is as follows: during the process of retrieving the target vector data of the current layer from its own memory based on the identifier of the current layer and performing the current layer vector retrieval calculation with the query vector data, if the number of the cumulative missing target vector data of the current layer is greater than the missing threshold, batch loading is triggered.

[0024] Optionally, during the process of the computation layer retrieving the target vector data of the current layer from its own memory through the caching module according to the identifier of the current layer and performing the current layer vector retrieval calculation with the query vector data, the intermediary layer preloads the target vector data of the next layer.

[0025] Optionally, the intermediary layer includes a memory optimization module, which determines the memory capacity of the computing layer according to the following heuristic algorithm when deploying the browser-oriented vector retrieval system to the device:

[0026] The memory optimization module performs benchmark tests using the maximum available memory capacity to obtain key parameters, which include at least the number of accesses, the time taken for a single query, and the total query time.

[0027] The memory optimization module determines the access count threshold based on the key parameters. The access count threshold represents the number of accesses that, when the memory capacity is reduced, will not cause the query time to exceed the time threshold.

[0028] The memory optimization module iteratively tests to establish a quantitative relationship between memory capacity and access count. Based on linear interpolation and the access count threshold, it predicts the next memory capacity test point. Under the condition of iterative convergence, it determines the memory capacity of the computing layer.

[0029] The memory capacity value of the computing layer represents the minimum memory capacity under the condition that it is less than the time consumption threshold. The iterative convergence condition includes the current access count test value being greater than or equal to the access count threshold or the memory capacity being reduced to the minimum feasible capacity value. The current access count test value is determined based on the current memory capacity test value.

[0030] Optionally, the quantization relationship is a linear relationship; the memory optimization module iteratively tests to establish a quantization relationship between memory capacity and access count, predicts the next memory capacity test point based on linear interpolation and the access count threshold, and determines the memory capacity of the computing layer under the condition of iterative convergence, including:

[0031] The memory optimization module sets the memory capacity to 0, determines the extreme point of access count, and randomly selects a current memory capacity test point to obtain the current access count test value point.

[0032] The memory optimization module uses the line connecting the extreme point of the access count and the current access count test value as a linear relationship between the current memory capacity and the access count.

[0033] The memory optimization module determines the next memory capacity test point based on the linear relationship between the current memory capacity and the number of accesses, as well as the access number threshold.

[0034] The memory optimization module performs a test based on the next memory capacity test point to obtain the next access count test value point.

[0035] The memory optimization module repeats the above steps, iteratively testing to establish a linear relationship between memory capacity and access count until the current access count test value is greater than or equal to the access count threshold, at which point the iteration ends and the previous memory capacity test point is taken as the memory capacity of the computing layer; or, if the current memory capacity test point is reduced to the minimum feasible capacity value, the iteration ends and the current memory capacity test point is taken as the memory capacity of the computing layer.

[0036] Optionally, the storage layer includes a first key-value table and a second key-value table, wherein the first key-value table uses the identifier of the vector data as the key and the original data corresponding to the vector data as the value; and the second key-value table uses the identifier of the vector data as the key and the vector embedding corresponding to the vector data as the value.

[0037] The storage layer stores all the vector data in key-value pairs according to the first key-value table and the second key-value table.

[0038] The embodiments of this application have the following advantages:

[0039] In this embodiment, the vector retrieval system for browser environments includes a task scheduling module and a hierarchical data management module. After the task scheduling module receives the user's query vector data, it schedules the hierarchical data management module to execute the vector retrieval task. Since the hierarchical data management module adopts a three-layer data management architecture, including a computation layer storing a first part of frequently accessed vector data, an intermediary layer storing a second part of frequently accessed vector data, and a storage layer storing all vector data, when responding to a vector retrieval task, the computation layer can directly retrieve vector data and query vector data from its own memory for vector retrieval calculation, without needing to access the storage layer every time. The computation layer only retrieves the missing target vector data and query vector data in batches from the intermediary layer and / or from the storage layer through the intermediary layer for vector retrieval calculation when the target vector data is missing from its own memory. Thus, based on the three-layer data management architecture, the synchronous / asynchronous execution model is effectively coordinated, ensuring seamless integration of high-performance computing and browser storage, reducing the number of accesses to the storage layer, avoiding the loading of redundant vector data, and overcoming the lack of storage access efficiency. Furthermore, by adding a computation layer to the system for vector retrieval calculations (i.e., accelerating computationally intensive operations through the computation layer), the system overcomes the parsing and execution bottleneck of existing browsers using JavaScript engines, thus improving vector retrieval performance. Therefore, this system enables efficient vector retrieval in browser-based environments. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of a vector retrieval system for a browser environment provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of a heuristic memory optimization mechanism provided in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of another vector retrieval system for browser environments provided in this application embodiment. Detailed Implementation

[0044] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] Among related technologies, the Hierarchical Navigable Small World (HNSW) algorithm is mainly used as the indexing architecture for browser-side vector retrieval engines. While this approach has initially achieved vector retrieval functionality in a browser environment, it still faces at least the following technical bottlenecks in actual deployment: 1) Computational performance bottleneck: The parsing and execution characteristics of browser JavaScript engines lead to inefficient computationally intensive operations; HNSW queries require a large number of computational operations such as vector similarity and sorting. Experimental data shows that the computational latency required for a single HNSW query by existing browser-side vector retrieval engines exceeds 100 milliseconds. This latency becomes a performance bottleneck for web applications, making it difficult to meet the real-time interactive requirements of browsers.

[0046] 2) Storage access efficiency bottleneck: The heuristic data loading mechanism with a fixed prefetch size loads missing data from IndexedDB storage into memory, leading to significant redundant data loading issues. Experimental data shows that when the memory size is less than 98% of the total data size, the redundancy rate of prefetched data exceeds 80%, resulting in a large amount of invalid I / O (Input / Output) latency. Simultaneously, frequent IndexedDB storage access (such as a single retrieval triggering more than ten or even hundreds of IndexedDB storage accesses) causes access latency to surge to the second level, severely impacting user experience.

[0047] 3) Rigid memory resource utilization: Existing browser-side vector retrieval engines employ a static memory allocation strategy, which cannot adapt to the dynamic resource constraints of the browser environment. This typically manifests as the predefined cache size failing to automatically adjust according to the device's memory capacity, easily leading to memory overflows and browser crashes on low-end mobile devices. This static memory allocation strategy results in severe resource waste or performance degradation when the system is deployed across multiple devices.

[0048] To overcome the limitations of related technologies, this application provides a vector retrieval system for browser environments. This system employs a three-layer data management architecture. When performing vector retrieval tasks, the computation layer can directly retrieve vector data and query vector data from its own memory for vector retrieval calculations, eliminating the need to access the storage layer every time. The computation layer only retrieves the missing target vector data and query vector data in batches from the intermediary layer and / or from the storage layer through the intermediary layer when target vector data is missing from its own memory. This effectively coordinates synchronous / asynchronous execution models, ensuring seamless integration of high-performance computing and browser storage, reducing the number of accesses to the storage layer, avoiding the loading of redundant vector data, and overcoming the inefficiency of storage access. Furthermore, performing vector retrieval calculations through the computation layer overcomes the parsing and execution bottleneck of existing browsers using JavaScript engines, improving vector retrieval calculation performance. Therefore, efficient vector retrieval in browser environments is achieved.

[0049] In some embodiments, only when the data loading conditions are met is the batch acquisition of missing target vector data from the intermediary layer and / or from the storage layer through the intermediary layer triggered to perform vector retrieval calculations with the query vector data, dynamically adjusting the timing of vector data loading to further avoid redundant data prefetching.

[0050] In some embodiments, when deploying a browser-oriented vector retrieval system to a device, the memory capacity of the computing layer is determined by a heuristic algorithm, thereby dynamically adjusting the memory capacity according to device performance and saving memory usage while ensuring query latency.

[0051] The vector retrieval system for browser environments provided in this application will now be described with reference to the accompanying drawings.

[0052] Reference Figure 1 As shown, Figure 1 This is a schematic diagram of a vector retrieval system for a browser environment provided in an embodiment of this application. For example... Figure 1 As shown, this vector retrieval system for browser environments includes a task scheduling module and a hierarchical data management module, wherein:

[0053] The task scheduling module receives the user's query vector data and schedules the hierarchical data management module to execute the vector retrieval task. The query vector data represents the query information provided by the user.

[0054] The hierarchical data management module includes a computation layer storing a first part of high-frequency access vector data, an intermediary layer storing a second part of high-frequency access vector data, and a storage layer storing all vector data. The high-frequency access vector data represents vector data that has been accessed more than a threshold number of times within a target time period.

[0055] In response to the vector retrieval task, the hierarchical data management module performs vector retrieval calculations by retrieving target vector data and query vector data from its own memory, and by retrieving missing target vector data and query vector data in batches from the intermediary layer and / or from the storage layer through the intermediary layer when target vector data is missing from its own memory.

[0056] In this embodiment, the computation layer is responsible for performing efficient vector retrieval computations on the browser side (e.g., HNSW graph traversal, vector similarity calculation, and candidate set sorting). The computation layer is compiled from the core computational logic of the HNSW algorithm (e.g., vector similarity calculation, candidate set sorting, etc.) (e.g., compiled using the Emscripten toolchain, an open-source compiler toolchain). Compared to using a JavaScript engine, this computation layer can achieve near-native code performance. In some embodiments, the computation layer is implemented based on WebAssembly, where WebAssembly (i.e., Wasm) is a code compilation format that enables programs written in high-level languages ​​to run in web browsers and has near-native execution speed. The intermediary layer uses a JavaScript intermediary layer, which acts as a bridge between the computation layer and the storage layer, handling complex data serialization, asynchronous task scheduling, and memory monitoring. The storage layer refers to the IndexedDB storage layer, responsible for storing the original vector data, i.e., storing all vector data.

[0057] To address the limitations of computational performance, insufficient memory capacity, and inefficient storage access in browser environments, this system constructs a three-tiered data management architecture (i.e., a hierarchical data management module). The computational layer caches frequently accessed vector data (the first part of frequently accessed vector data) in memory, and an intermediate layer manages additional vector data that the computational layer cannot cache (the second part of frequently accessed vector data). IndexedDB serves as persistent storage (the storage layer), storing the complete index structure and vector data—that is, storing all vector data, including the first part of frequently accessed vector data, the second part of frequently accessed vector data, and the remaining vector data. This three-tiered data management architecture effectively solves the problems of limited computational performance, insufficient memory capacity, and inefficient storage access in browser environments.

[0058] The task scheduling module is responsible for managing the hierarchical data management module. During use, for the user's query vector data (i.e., the user's input and output), the task scheduling module schedules the computing layer to continuously execute high-density vector retrieval calculations. Specifically, performing vector retrieval calculations between the target vector data and the query vector data can be understood as: calculating the similarity between the target vector data and the query vector data, and sorting them according to the similarity calculation results, thereby filtering out the vector data that meets the user's needs based on the sorting.

[0059] Specifically, since the computation layer stores the first part of frequently accessed vector data, it prioritizes retrieving the corresponding target vector data from its own memory and performing vector retrieval calculations with the query vector data. If all target vector data can be retrieved from its own memory and used for vector retrieval calculations with the query vector data, there is no need to access the vector data in the intermediate layer and storage layer. If the computation layer lacks target vector data (i.e., the corresponding target vector data cannot be retrieved from its own memory), it retrieves the missing target vector data from the intermediate layer or storage layer in batches and performs vector retrieval calculations with the query vector data. Retrieving the correct target vector data in batches reduces the number of accesses to the storage layer and avoids loading redundant vector data. Furthermore, the vector retrieval calculations of the computation layer and the storage accesses of the storage layer are executed asynchronously, maximizing the utilization of computational resources and I / O bandwidth.

[0060] Understandably, in this system, vector data in the computation layer, intermediary layer, and storage layer are all stored using a text-vector separation method. Data is retrieved using a unique identifier (ID), thus reducing storage overhead. Furthermore, large index files (i.e., files exceeding a certain size threshold) are indexed in fragments. In addition, the system uses a pluggable cache eviction strategy (e.g., default FIFO) to balance the efficiency of accessing hot and cold data. These optimizations, working in conjunction with the core algorithm, enable efficient vector retrieval.

[0061] Through the above implementation process, the system adopts a three-tier data management architecture, effectively coordinating synchronous / asynchronous execution models, ensuring seamless integration between high-performance computing and browser storage, reducing the number of accesses to the storage layer, avoiding the loading of redundant data, and overcoming the lack of storage access efficiency. Furthermore, by adding a computing layer to the system for vector retrieval calculations (i.e., accelerating computationally intensive operations through the computing layer), the system overcomes the parsing and execution bottleneck of existing browsers using JavaScript engines, improving vector retrieval performance. Therefore, this system enables efficient vector retrieval in the browser environment.

[0062] The following sections provide a detailed description of the vector retrieval system for browser environments provided in this application, specifically through Section 1.1 (three-layer data management architecture), Section 1.2 (phased lazy loading mechanism), and Section 1.3 (heuristic memory optimization mechanism).

[0063] 1.1 Three-tier data management architecture:

[0064] In one optional embodiment, the computation layer is implemented based on WebAssembly. The computation layer includes a hierarchical index structure and a caching module. Each layer of the hierarchical index structure consists of identifiers for vector data. Specifically, the computation layer retrieves target vector data from its own memory and performs vector retrieval calculations with the query vector data. This includes: the caching module of the computation layer, according to the hierarchical index structure, sequentially retrieving the target vector data of the current layer from its own memory based on the identifier of the current layer, and performing vector retrieval calculations with the query vector data for the current layer.

[0065] In this embodiment, the computation layer is implemented based on WebAssembly. Since WebAssembly is limited to 32-bit addressing and cannot directly access IndexedDB, the vector data in the storage layer can be retrieved through an intermediary layer via the three-level data management interface of the hierarchical data management module. The computation layer includes a caching module that maintains a cross-level cache. The caching module uses the vector data identifier (i.e., the vector data ID) as the key to cache each set of loaded vector data. The computation layer organizes all vector data together through a hierarchical index structure, which includes multiple levels, each composed of vector data identifiers. The computation layer performs vector retrieval and computation layer by layer.

[0066] Understandably, in a hierarchical index structure, the earlier levels (e.g., level 1) typically correspond to identifiers for frequently accessed vectors. Therefore, when performing vector retrieval according to the hierarchical index structure, the corresponding target vector data and query vector data can be preferentially retrieved from the computed memory for vector retrieval calculation. Furthermore, the hierarchical index structure is adjustable; the computation layer can adjust the identifiers of each level in the hierarchical index structure based on the access frequency of the vector data.

[0067] When the hierarchical data management module responds to a vector retrieval task, the computation layer, through the caching module, retrieves the target vector data of the current layer from its own memory according to the hierarchical index structure and the identifier of the current layer, and performs vector retrieval calculations for the current layer with the query vector data. If the computation layer lacks the target vector data of the current layer in its own memory, it retrieves the missing target vector data of the current layer in batches from the intermediary layer and / or from the storage layer through the intermediary layer.

[0068] Further, the intermediary layer includes a first application programming interface; when the computation layer lacks target vector data in its own memory, it batch retrieves the missing target vector data from the intermediary layer and / or from the storage layer through the intermediary layer, along with the query vector data, to perform vector retrieval calculations, including:

[0069] Step A1: When the computing layer is missing the target vector data of the current layer in its own memory, it notifies the intermediary layer to check the heap cache through the first application programming interface of the intermediary layer, retrieves the missing target vector data of the current layer from the heap cache of the intermediary layer, and performs vector retrieval calculation with the query vector data;

[0070] Step A2: If the heap cache of the intermediary layer does not contain missing target vector data of the current layer, the intermediary layer retrieves the missing target vector data of the current layer from the storage layer and performs vector retrieval calculation with the query vector data.

[0071] In this embodiment of the application, when the computing layer performs vector retrieval calculations, it follows a strict data access order to obtain the corresponding target vector data. That is, it first obtains the target vector data from its own memory, then checks the heap cache of the intermediary layer, and finally triggers the access of the storage layer.

[0072] Specifically, for each level in the hierarchical index structure, the target vector data for the current level is first retrieved from the memory of the computation layer and used to perform vector retrieval calculations with the query vector data. If the computation layer itself lacks the target vector data for the current level, step A1 is executed, notifying the intermediary layer via the first application programming interface (i.e., get() API, where API refers to Application Programming Interface) to check the push cache to retrieve the missing target vector data for the current level and use it to perform vector retrieval calculations with the query vector data. If all the missing target vector data for the current level can be retrieved from the heap cache of the intermediary layer, the vector retrieval for the current level can be terminated. If there is still missing target vector data, step A2 is executed to retrieve the missing target vector data for the current level from the storage layer and use it to perform vector retrieval calculations with the query vector data. The intermediary layer acts as a bridge between the computation layer and the storage layer; when the computation layer needs to access the storage layer, it accesses the storage layer through the intermediary layer.

[0073] Through the above implementation process, vector data is stored based on a three-tier data management architecture, and target vector data and query vector data are retrieved and calculated according to the data access order. This multi-level caching mechanism significantly reduces the overhead of external storage access.

[0074] In one optional embodiment, during the process of the computation layer retrieving the target vector data of the current layer from its own memory through the caching module according to the identifier of the current layer and performing the current layer vector retrieval calculation with the query vector data, the intermediary layer preloads the target vector data of the next layer.

[0075] Specifically, while scheduling the computation layer to perform vector retrieval for the current layer, the task scheduling module simultaneously preloads the target vector data for the next layer asynchronously through the intermediary layer (for example, loading the target vector data for the next layer from the storage layer). This improves the efficiency of obtaining vector data and enables efficient vector retrieval in a browser environment.

[0076] In an optional embodiment, the intermediary layer further includes a second application programming interface and a third application programming interface;

[0077] The computation layer updates the vector data in its own memory to the intermediary layer and the storage layer for storage through the second application programming interface of the intermediary layer;

[0078] The computation layer, through the third application programming interface of the intermediary layer, suspends the target task to the end of the event queue, waiting for the intermediary layer to read the missing target vector data of the current layer from its own heap cache and / or from the storage layer. Once the intermediary layer has completed reading the missing target vector data of the current layer from its own heap cache and / or from the storage layer, the computation layer continues to execute the target task. The target task represents obtaining the target vector data of the current layer and performing vector retrieval calculation of the current layer with the query vector data.

[0079] In this embodiment, the intermediary layer acts as a bridge between the computation layer and the storage layer. It can also update vector data in the computation layer's memory to the storage layer for storage via the intermediary layer's second application programming interface (i.e., the store() API). Thus, the data transfer function implemented by the intermediary layer through the first and second application programming interfaces allows the computation layer to obtain vector data from the storage layer via the first application programming interface, and the computation layer to store vector data in the storage layer via the second application programming interface.

[0080] The intermediary layer can also perform model coordination functions, that is, the synchronous computation model of the computation layer and the asynchronous I / O of the storage layer are coordinated through the event loop of the intermediary layer. Specifically, when the computation layer is waiting for the intermediary layer to retrieve the missing target vector data of the current layer from the storage layer, it suspends the target task to the end of the event queue through the third application programming interface (i.e., wait() API), and after the intermediary layer finishes reading the missing target vector data of the current layer from the storage layer, it notifies the computation layer to resume the execution of the target task through a shared signal.

[0081] Through the above implementation process, by deploying application programming interfaces (e.g., first program programming interface, second program programming interface, and third program programming interface) in the intermediary layer, the intermediary layer acts as a bridge between synchronous and asynchronous execution models. It coordinates the synchronous vector retrieval computation of the computation layer and the asynchronous I / O operation of the computation layer through a shared signal mechanism. When a cache miss occurs in the computation layer (i.e., the target vector data is missing from the memory of the computation layer), the intermediary layer is responsible for loading data in batches from the storage layer and ensuring the efficient execution of the computation task through event loop scheduling.

[0082] In one optional embodiment, the storage layer includes a first key-value table and a second key-value table, wherein the first key-value table uses the identifier of the vector data as the key and the original data corresponding to the vector data as the value; and the second key-value table uses the identifier of the vector data as the key and the vector embedding corresponding to the vector data as the value.

[0083] The storage layer stores all the vector data in key-value pairs according to the first key-value table and the second key-value table.

[0084] In this embodiment, for all vector data, the original data corresponding to the vector data is stored as a value through a first key-value table, and the original data corresponding to the vector data is stored as a value through a second key-value table. This separate storage method can accelerate data retrieval and data transmission in the storage layer.

[0085] 1.2 Phased Lazy Loading Mechanism:

[0086] In an optional embodiment, when the computation layer lacks target vector data in its own memory, it batch retrieves the missing target vector data from the intermediary layer and / or from the storage layer through the intermediary layer, along with the query vector data, to perform vector retrieval calculations, including:

[0087] When the computing layer lacks target vector data in its own memory and meets the data loading conditions, it batch retrieves the missing target vector data and the query vector data from the intermediary layer and / or from the storage layer through the intermediary layer to perform vector retrieval calculations. The data loading conditions are used to trigger the batch loading of the missing target vector data in the computing layer's own memory.

[0088] In this embodiment, considering that traditional prefetching strategies can lead to the loading of a large amount of redundant data when memory is limited, resulting in a decrease in the vector retrieval performance of the system, this embodiment proposes a phased lazy loading mechanism to dynamically adjust the timing of data loading. Specifically, during the vector retrieval calculation process in the computation layer, if the target vector data is missing from its own memory, the missing target vector data and query vector data will not be immediately retrieved from the intermediary layer or storage layer for vector retrieval calculation. Instead, the missing target vector data in the computation layer's own memory will only be batch loaded when the data loading conditions are met, thereby improving storage access efficiency.

[0089] Furthermore, the storage layer includes a data access module, which manages the data loading conditions, including a first data loading condition and a second data loading condition.

[0090] The first data loading condition is as follows: during the process of retrieving the target vector data of the current layer from its own memory according to the identifier of the current layer and performing the current layer vector retrieval calculation with the query vector data, if the number of missing target vector data of the current layer is less than or equal to the missing threshold, the loading of the missing target vector data corresponding to the current layer is temporarily suspended, and after retrieving the target vector data of the current layer from its own memory according to the identifier of the current layer and performing the current layer vector retrieval calculation with the query vector data, batch loading is triggered;

[0091] The second data loading condition is as follows: during the process of retrieving the target vector data of the current layer from its own memory based on the identifier of the current layer and performing the current layer vector retrieval calculation with the query vector data, if the number of the cumulative missing target vector data of the current layer is greater than the missing threshold, batch loading is triggered.

[0092] In this embodiment of the application, during the vector retrieval process, if the computing layer lacks target vector data in its own memory, it will only trigger batch loading if the first data loading condition or the second data loading condition is met. Batch loading refers to obtaining the missing target vector data in batches from the intermediary layer and / or from the storage layer through the intermediary layer.

[0093] Specifically, the first data loading condition can also be understood as the inter-layer loading phase. During the current layer vector retrieval calculation, as long as the cumulative number of missing target vector data for the current layer is less than or equal to the missing threshold, the missing target vector data is temporarily ignored. After completing the vector retrieval calculation for the current layer in memory, all missing target vector data for that layer are loaded uniformly to ensure the correct entry point for the next layer. For example, during the current layer vector retrieval calculation, if the target vector data that the current layer needs to access is... ,in If a memory miss occurs, loading will be deferred. Continue using memory Complete the vector retrieval calculation (search) for the current layer, and then batch load the vectors after the vector retrieval calculation for that layer is finished. And its neighbors. In this way, triggering batch loading through the first data loading condition can reduce a lot of redundant loading, while ensuring that the entry point of the next layer is correct.

[0094] The second data loading condition can also be understood as the in-layer loading phase. During the vector retrieval calculation of the current layer, if the cumulative number of missing target vector data in the current layer exceeds the missing threshold, batch loading is immediately triggered to avoid deviation of the query path. For example, if the missing threshold is 100, and 201 missing target vector data accumulates during the vector retrieval calculation of a certain layer, the vector retrieval calculation is interrupted, and all missing target vector data is directly retrieved from the intermediate layer and / or from the storage layer through the intermediate layer, avoiding result deviation due to broken search paths.

[0095] Through the above implementation process, when the computing layer lacks target vector data in its own memory, it triggers the batch loading of the missing target vector data through data loading conditions (i.e., the first data loading condition and the second data loading condition). In this way, while ensuring query accuracy, the frequency of access to the storage layer is reduced.

[0096] 1.3 Heuristic memory optimization mechanisms:

[0097] In an optional embodiment, the intermediary layer includes a memory optimization module that determines the memory capacity of the computing layer according to the following heuristic algorithm when deploying the browser-oriented vector retrieval system to a device:

[0098] Step B1: The memory optimization module performs a benchmark test with the maximum available memory capacity to obtain key parameters, which include at least the number of accesses, the time taken for a single query, and the total query time.

[0099] Step B2: The memory optimization module determines the access count threshold based on the key parameters. The access count threshold represents the number of accesses that, when the memory capacity is reduced, will not cause the query time to exceed the time threshold.

[0100] Step B3: The memory optimization module iteratively tests to establish a quantitative relationship between memory capacity and access count. Based on linear interpolation and the access count threshold, it predicts the next memory capacity test point. Under the condition of iterative convergence, it determines the memory capacity of the computing layer.

[0101] The memory capacity value of the computing layer represents the minimum memory capacity under the condition that it is less than the time consumption threshold. The iterative convergence condition includes the current access count test value being greater than or equal to the access count threshold or the memory capacity being reduced to the minimum feasible capacity value. The current access count test value is determined based on the current memory capacity test value.

[0102] In this embodiment, the optimal memory threshold is adaptively determined through iterative testing and mathematical model calculation to minimize memory usage while ensuring query performance. Specifically, the initial testing phase of step B1 is first executed, using the maximum available memory capacity (e.g., 256MB of memory capacity allowed by the browser environment) for benchmark testing to obtain key parameters. These key parameters are used to subsequently calculate the optimal memory capacity, ensuring that the system reduces memory usage without causing a sharp drop in query performance due to excessive cache compression. The key parameters include at least the number of accesses, the time spent on a single query, and the total query time. The number of accesses refers to the number of times the storage layer is accessed during a single query (vector retrieval calculation). The time spent on a single query refers to the average time spent loading data from the storage layer. The total query time refers to the complete time from the start of the query to the return of the result. Additionally, the key parameter can also include the query path length, i.e., the number of vector data items traversed.

[0103] Next, the iterative optimization phases B2 and B3 are executed. Based on the initial test data, the system dynamically adjusts the memory size using a mathematical model and linear interpolation until the optimal memory capacity is found. In step B2, the memory optimization module determines the access count threshold based on key parameters. An access count can be determined using both a proportional threshold method and an absolute latency threshold method, and the larger of these two methods is used as the access count threshold to ensure that query time remains controllable. For example, if the access count is 3, the single query time is 32 milliseconds, and the total query time is 186 milliseconds, the process of determining the access count using the proportional threshold method is as follows: set the access time to not exceed a proportion of the total query time (e.g., 0.8), then the access count is... This means the system allows a maximum of 4 to 5 accesses. The process of determining the number of accesses using a delay threshold method is as follows: Set the access time to not exceed the target query time (the target query time is less than the total query time, e.g., the target query time is 100 milliseconds), then the number of accesses is... That is, the system allows a maximum of 3 accesses; ultimately, the access threshold of 5 times can be determined.

[0104] In step B3, a heuristic approach is used to dynamically determine the optimal memory capacity for the computation layer, avoiding resource waste or performance degradation caused by static allocation. The core idea is to model the query process as a black-box system, establishing a quantitative relationship between memory capacity and access frequency (representing query latency) through iterative testing, and intelligently adjusting the memory allocation strategy based on this relationship. Specifically, starting from the maximum memory capacity, the memory percentage is gradually reduced while monitoring the number of accesses to the storage layer. Linear interpolation is used to approximate the optimal memory point, ensuring that access time does not exceed a threshold (e.g., 20% of total access time or an absolute value of 100ms). This allows the system to adapt to different device performance, saving an average of 31% of memory usage in browsers while maintaining stable query latency.

[0105] Furthermore, the quantification relationship is a linear relationship, and step B3 may include sub-steps B31 to B35:

[0106] Step B31: The memory optimization module sets the memory capacity to 0, determines the extreme point of the access count, and randomly selects a current memory capacity test point for testing to obtain the current access count test value point;

[0107] When the memory capacity is 0, theoretically, each data access requires external data access. This means that the computation layer retrieves the corresponding target vector data from the intermediary layer or storage layer during each layer's vector retrieval calculation. The current access count test value refers to the actual number of accesses at the current memory capacity test point.

[0108] Step B32: The memory optimization module uses the line connecting the extreme point of the access count and the current access count test value as a linear relationship between the current memory capacity and the access count;

[0109] For example, the current memory capacity (C) and the number of accesses (C) The linear relationship between them can be expressed as:

[0110] ,

[0111] Among them, the slope k reflects the degree of influence of changes in memory capacity on the number of accesses, and the intercept b is related to the deployed device, with different values ​​for b corresponding to different devices.

[0112] Step B33: The memory optimization module determines the next memory capacity test point based on the linear relationship between the current memory capacity and the number of accesses, as well as the access count threshold.

[0113] Specifically, by substituting the access count threshold into the linear relationship between the current memory capacity and the access count, we can obtain the next memory capacity test point. That is, the memory capacity at the intersection of the line connecting the extreme point of the access count and the current access count test value point and the line where y equals the access count threshold can be used as the next memory capacity test point.

[0114] Step B34: The memory optimization module performs a test based on the next memory capacity test point to obtain the next access count test value point;

[0115] Step B35: The memory optimization module repeats the above steps, iteratively testing to establish a linear relationship between memory capacity and access count until the current access count test value is greater than or equal to the access count threshold, then the iteration ends and the previous memory capacity test point is taken as the memory capacity of the computing layer; or, if the current memory capacity test point is reduced to the minimum feasible capacity value, the iteration ends and the current memory capacity test point is taken as the memory capacity of the computing layer.

[0116] Specifically, if the current access count test value is less than the access count threshold, it means that reducing memory at this point will not cause the query time to exceed the threshold, and the iterative test continues to establish a linear relationship between memory capacity and access count. If the current access count test value is greater than or equal to the access count threshold, it means that reducing memory at this point will affect query time (query latency), and the iteration stops. Therefore, the test is rolled back to the previous capacity test point, and the previous memory capacity test point is used as the memory capacity of the computing layer. In this way, through the memory rollback mechanism, when the current access count test value is greater than or equal to the access count threshold, the memory capacity is automatically restored to the previous level of safe memory value, ensuring service reliability. Furthermore, when the current memory capacity test point decreases to the minimum feasible capacity value, it means that the optimal memory capacity has been found, and the current memory capacity test point is used as the content capacity of the computing layer.

[0117] like Figure 2 As shown, Figure 2 This is a schematic diagram of a heuristic memory optimization mechanism provided in an embodiment of this application. Figure 2 This diagram illustrates the process of establishing a quantitative relationship between memory capacity and access count through iterative testing, based on linear interpolation combined with an access count threshold to predict the next memory capacity test point. Solid lines represent the best quantitative relationship between memory capacity and access count, while dashed and dotted lines represent the linear relationship between them. This represents the maximum number of accesses, and N represents the maximum memory capacity.

[0118] Specifically, firstly based on the maximum number of visits Establish the extreme point A for the number of visits, and the random content capacity. Establish test value point for current access count The extreme point A of the number of visits and the test point of the current number of visits are compared. The line connecting the two represents the linear relationship between the current memory capacity and the number of accesses; if the current access count test value is less than the access count threshold... Get the connection and intersection memory capacity This will serve as the next memory capacity test point; according to Determine the next test value point for the number of visits. The extreme point A of the number of visits and the test point of the next number of visits. The connection between the two represents a linear relationship between the next memory capacity and the number of accesses. The connection is then used to obtain the relationship between the two. intersection memory capacity This serves as the next memory capacity test point. The above steps are repeated until the iterative convergence condition is met, thus determining the memory capacity of the computation layer.

[0119] Through the above implementation process, a heuristic method is used to dynamically determine the optimal memory capacity of the computing layer, avoiding resource waste or performance degradation caused by static allocation. Therefore, the system can automatically adjust memory usage based on device performance, browser type, and dataset size. A mathematical model ensures that query time remains controllable, preventing performance drops due to memory compression. Compared to existing fixed memory allocation strategies, this saves 7% to 39% of memory and improves compatibility with low-end devices. This ensures that the system maintains efficient and stable retrieval performance under different hardware environments, solving the resource waste or performance fluctuation problems caused by static memory allocation in traditional solutions.

[0120] like Figure 3 As shown, Figure 3 This is a schematic diagram of another vector retrieval system for a browser environment provided in this application embodiment. The system includes a task scheduling module and a hierarchical data management module. The hierarchical data management module is a three-layer data management architecture, including a computation layer, an intermediary layer, and a storage layer. The computation layer includes a caching module for maintaining cross-layer caches. The intermediary layer includes a memory optimization module. When deploying the system to a device, the memory optimization module adjusts the content capacity of the computation layer using heuristic algorithms and detects query latency (access time), using as little memory as possible while meeting preset time thresholds. The storage layer includes a data access module that uses lazy loading technology to manage data loading conditions, reducing the frequency of storage access in the storage layer.

[0121] Specifically, after the task scheduling module receives the user's query vector data, the scheduling hierarchical data management module executes the vector retrieval task. In response to this task, the hierarchical data management module first retrieves the target vector data of the current layer from its own memory according to the hierarchical index structure and the identifier of the current layer, and then performs vector retrieval calculations with the query vector data. If the computation layer lacks target vector data in its own memory but meets the data loading conditions, it retrieves the missing target vector data in batches from the intermediary layer and / or from the storage layer through the intermediary layer, and performs vector retrieval calculations with the query vector data, thereby achieving efficient vector retrieval in the browser environment.

[0122] The vector retrieval system for browser environments provided in this application has the following significant advantages compared with the prior art:

[0123] Advantage 1: Significantly Improved Computational Performance. This system accelerates computationally intensive operations (such as vector similarity calculation and sorting) through a computational layer (WebAssembly), overcoming the parsing and execution bottlenecks of traditional JavaScript engines. Experimental data shows that, under the same hardware environment, the system's P99 (P99 refers to the performance metric that measures 99% of requests within the maximum latency in the service response time distribution) query latency is reduced by 70 to 743 times, optimizing from seconds to milliseconds, meeting real-time interaction requirements.

[0124] Advantage 2: Optimized storage access efficiency. Employing a phased lazy loading mechanism, the system dynamically adjusts data loading timing, avoiding redundant data prefetching. Compared to existing fixed prefetching mechanisms, this system reduces invalid disk I / O by more than 50% when memory coverage is below 98%, enabling retrieval of large datasets (such as the 7.5GB Wiki-480k dataset) to maintain a 10ms response time.

[0125] Advantage 3: Improved memory resource utilization. Based on a heuristic memory optimization mechanism, the system can dynamically adjust the cache threshold according to device performance, saving 7% to 39% of memory usage while ensuring query latency (e.g., <100ms). Furthermore, the text-vector separation storage design further reduces single-entry memory overhead, enabling the system to run stably even on low-end devices.

[0126] Advantage 4: Enhanced cross-device compatibility. Through a three-tiered data management architecture based on hierarchical data management modules, the system effectively coordinates synchronous / asynchronous execution models, ensuring seamless integration of high-performance computing and browser storage. Experiments demonstrate that the system maintains stable performance on Chrome, Firefox, and Safari (referring to three different browsers) and various hardware (PCs / mobile devices), supporting datasets expanded by more than 8.5 times.

[0127] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0128] This application describes embodiments of methods and apparatus according to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0132] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0133] The above provides a detailed description of a vector retrieval system for browser environments provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A vector retrieval system for a browser environment, characterized by Comprise a task scheduling module and a hierarchical data management module, wherein: The task scheduling module receives a user's query vector data and schedules the hierarchical data management module to execute a vector retrieval task, the query vector data representing query information provided by the user; The computing layer is implemented based on WebAssembly, the hierarchical data management module comprises a computing layer storing a first part of high-frequency access vector data, an intermediate layer storing a second part of high-frequency access vector data, and a storage layer storing all vector data, wherein the high-frequency access vector data represents vector data that is accessed more than a threshold number of times within a target time period; The hierarchical data management module responds to the vector retrieval task, the computing layer obtains target vector data from its own memory and performs vector retrieval calculation with the query vector data, and In the case that the computing layer lacks target vector data in its own memory, it obtains the missing target vector data from the intermediate layer and / or the storage layer in bulk and performs vector retrieval calculation with the query vector data, including: in the case that the computing layer lacks target vector data in its own memory and meets a data loading condition, it obtains the missing target vector data from the intermediate layer and / or the storage layer in bulk and performs vector retrieval calculation with the query vector data, the data loading condition being used to trigger bulk loading of the missing target vector data in the computing layer's own memory; The intermediate layer comprises a memory optimization module, which determines the memory capacity of the computing layer according to the following heuristic algorithm when deploying the browser-oriented vector retrieval system to a device: The memory optimization module performs a benchmark test with the maximum available memory capacity to obtain key parameters, including at least the number of accesses, the time consumed by a single query, and the total query time; The memory optimization module determines an access threshold value based on the key parameters, the access threshold value representing the number of accesses that meet the condition that reducing the memory capacity will not cause the query time to exceed a time threshold value; The memory optimization module iteratively tests to establish a quantitative relationship between the memory capacity and the number of accesses, predicts the next memory capacity test point based on linear interpolation combined with the access threshold value, and determines the memory capacity of the computing layer under the condition that the iterative convergence condition is met; The memory capacity value of the computing layer represents the minimum memory capacity under the condition that the time threshold value is met, the iterative convergence condition includes the current number of access test points being greater than or equal to the access threshold value or the memory capacity being reduced to the lowest feasible capacity value, and the current number of access test points is determined according to the current memory capacity test point.

2. The system of claim 1, wherein, The computing layer comprises a hierarchical index structure and a cache module, each layer of the hierarchical index structure being composed of identifiers of vector data; The computing layer obtains target vector data from its own memory and performs vector retrieval calculation with the query vector data, including: The cache module of the computing layer obtains target vector data of a current layer from the memory of the computing layer according to the hierarchical index structure and the identifier of the current layer, and performs vector retrieval calculation of the current layer with the query vector data.

3. The system of claim 2, wherein, The intermediary layer comprises a first application programming interface; in the case that the target vector data is missing in the memory of the computing layer, the computing layer obtains the missing target vector data from the storage layer through the intermediary layer and / or the intermediary layer to perform vector retrieval calculation with the query vector data, comprising: In the case that the target vector data of the current layer is missing in the memory of the computing layer, the computing layer informs the intermediary layer to check the heap cache through the first application programming interface of the intermediary layer, obtains the missing target vector data of the current layer from the heap cache of the intermediary layer, and performs vector retrieval calculation with the query vector data. In the case that the target vector data of the current layer is missing in the memory of the computing layer, the computing layer informs the intermediary layer to check the heap cache through the first application programming interface of the intermediary layer, obtains the missing target vector data of the current layer from the heap cache of the intermediary layer, and performs vector retrieval calculation with the query vector data.

4. The system of claim 3, wherein, The intermediary layer further comprises a second application programming interface and a third application programming interface; The computing layer updates the vector data in the memory of the computing layer to the intermediary layer and the storage layer for storage through the second application programming interface of the intermediary layer; The computing layer suspends a target task to the tail of the event queue through the third application programming interface of the intermediary layer, waits for the intermediary layer to read the missing target vector data of the current layer from the storage layer, and in the case that the intermediary layer completes reading the missing target vector data of the current layer from the storage layer, the computing layer continues to execute the target task; wherein the target task represents obtaining the target vector data of the current layer and performing vector retrieval calculation of the current layer with the query vector data.

5. The system of claim 1, wherein, The storage layer comprises a data access module, and the data access module is used to manage the data loading condition, and the data loading condition comprises a first data loading condition and a second data loading condition; The first data loading condition is that in the process of obtaining the target vector data of the current layer from the memory according to the identifier of the current layer and performing vector retrieval calculation of the current layer with the query vector data, in the case that the number of accumulated missing target vector data of the current layer is less than or equal to the missing threshold, the missing target vector data corresponding to the current layer is temporarily suspended, and after obtaining the target vector data of the current layer from the memory according to the identifier of the current layer and performing vector retrieval calculation of the current layer with the query vector data, batch loading is triggered. The second data loading condition is that in the process of obtaining the target vector data of the current layer from the memory according to the identifier of the current layer and performing vector retrieval calculation of the current layer with the query vector data, in the case that the number of accumulated missing target vector data of the current layer is greater than the missing threshold, batch loading is triggered.

6. The system according to any of claims 2-5, characterized in that, The intermediate layer preloads the target vector data of the next layer in the process of the current layer vector retrieval calculation between the target vector data of the current layer obtained from the memory of the intermediate layer by the buffer module according to the identifier of the current layer and the query vector data.

7. The system of claim 1, wherein, The quantization relationship is a linear relationship; the memory optimization module iteratively tests the quantization relationship between the memory capacity and the access frequency, predicts the next memory capacity test point based on linear interpolation combined with the access frequency threshold, and determines the memory capacity of the computing layer under the condition of meeting the iterative convergence condition, including: The memory optimization module determines the access frequency extreme point when the memory capacity is 0, and randomly determines a current memory capacity test point for testing to obtain a current access frequency test value point; The memory optimization module takes the line between the access frequency extreme point and the current access frequency test value point as the linear relationship between the current memory capacity and the access frequency; The memory optimization module determines the next memory capacity test point according to the linear relationship between the current memory capacity and the access frequency and the access frequency threshold; The memory optimization module tests according to the next memory capacity test point to obtain a next access frequency test value point; The memory optimization module repeats the above steps to iteratively test the linear relationship between the memory capacity and the access frequency until the current access frequency test value point is greater than or equal to the access frequency threshold, ends the iteration, and takes the last memory capacity test point as the memory capacity of the computing layer; or, in the case where the current memory capacity test point is reduced to the minimum feasible capacity value, the iteration is ended, and the current memory capacity test point is taken as the memory capacity of the computing layer.

8. The system of claim 1, wherein, The storage layer includes a first key-value table and a second key-value table, the first key-value table takes the identifier of the vector data as the key and the original data corresponding to the vector data as the value; the second key-value table takes the identifier of the vector data as the key and the vector embedding corresponding to the vector data as the value; The storage layer stores all vector data in the form of key-value pairs according to the first key-value table and the second key-value table.

Images