Vector retrieval system oriented to browser environment

Abstract

The invention provides a vector retrieval system oriented to a browser environment, relates to the technical field of vector retrieval, and aims to realize efficient vector retrieval oriented to the browser environment. The system comprises a task scheduling module and a hierarchical data management module, and the task scheduling module receives query vector data of a user and schedules the hierarchical data management module to execute a vector retrieval task; the hierarchical data management module comprises a calculation layer storing a first part of high-frequency access vector data, an intermediate layer storing a second part of high-frequency access vector data, and a storage layer storing all vector data; the hierarchical data management module responds to the vector retrieval task, the calculation layer obtains the target vector data and the query vector data from the memory of the calculation layer to perform vector retrieval calculation, and under the condition that the memory of the calculation layer lacks the target vector data, the query vector data is obtained; and obtaining missing target vector data and query vector data from the intermediate layer in batches and / or from the storage layer through the intermediate layer to perform vector retrieval calculation.

Description

Technical Field

[0001] The present application relates to the field of vector retrieval technology, and in particular to a vector retrieval system oriented to a browser environment. Background Art

[0002] With the rapid development of artificial intelligence (AI), vector search technology plays a key role in areas such as search enhancement generation, recommendation systems, and semantic search. In privacy-sensitive scenarios, browser-based vector search engines, capable of processing user data locally without uploading it to the cloud, have become a crucial technology for protecting user privacy in finance, healthcare, education, and other fields. However, as browsers are cross-platform and lightweight operating environments, resource-intensive vector search technology poses significant challenges to browser computing power, memory management, and storage access mechanisms.

[0003] Among related technologies, the Hierarchical Navigable SmallWorld (HNSW) algorithm is mainly used as the index architecture for browser-side vector search engines. This index architecture expands storage space to the device disk through IndexedDB (an index-based database) to address the browser's memory limitations. In addition, a prefetch mechanism is used to reduce the frequency of IndexedDB storage access, thereby reducing the total IndexedDB storage access latency and improving vector search speed.

[0004] While these solutions have initially achieved vector search capabilities in browser environments, they still face at least the following technical bottlenecks in actual deployment: First, computational performance. The parsing and execution characteristics of browser JavaScript (a lightweight, interpreted programming language) engines result in inefficient computationally intensive operations, making it difficult to meet the real-time interactive requirements of browsers. Second, storage access efficiency is a bottleneck. The heuristic data loading mechanism with a fixed prefetch size loads missing data from IndexedDB storage into memory, leading to significant redundant data loading. Therefore, achieving efficient vector search in browser environments is a pressing technical challenge. Summary of the Invention

[0005] In view of the above problems, embodiments of the present application provide a vector retrieval system for a browser environment, so as to overcome the above problems or at least partially solve the above problems.

[0006] The embodiment of the present application provides a browser-oriented vector search system, comprising a task scheduling module and a hierarchical data management module, wherein: The task scheduling module receives the query vector data of the user and schedules the hierarchical data management module to execute the vector retrieval task, and the query vector data represents the query information provided by the user; The hierarchical data management module includes a computing layer storing the first part of the high-frequency access vector data, an intermediary layer storing the second part of the high-frequency access vector data, and a storage layer storing all vector data. Among them, the high-frequency access vector data represents the vector data whose access times in the target time period are greater than the access times threshold; In response to the vector retrieval task, the computing layer of the hierarchical data management module obtains the target vector data from its own memory and performs vector retrieval calculation with the query vector data. And when the computing layer lacks the target vector data in its own memory, it batches and obtains the missing target vector data from the intermediary layer and / or through the intermediary layer from the storage layer to perform vector retrieval calculation with the query vector data.

[0007] Optionally, the computing layer is implemented based on WebAssembly. The computing layer includes a hierarchical index structure and a cache module, and each layer of the hierarchical index structure is composed of the identifiers of the vector data; The computing layer obtains the target vector data from its own memory and performs vector retrieval calculation with the query vector data, including: The cache module of the computing layer sequentially obtains the target vector data of the current layer from its own memory according to the identifiers of the current layer in accordance with the hierarchical index structure, and performs vector retrieval calculation of the current layer with the query vector data.

[0008] Optionally, the intermediary layer includes a first application programming interface; when the computing layer lacks the target vector data in its own memory, it batches and obtains the missing target vector data from the intermediary layer and / or through the intermediary layer from the storage layer to perform vector retrieval calculation with the query vector data, including: When the computing layer lacks the target vector data of the current layer in its own memory, it notifies the intermediary layer to check the heap cache through the first application programming interface of the intermediary layer, obtains the missing target vector data of the current layer from the heap cache of the intermediary layer, and performs vector retrieval calculation with the query vector data; When the missing target vector data of the current layer does not exist in the heap cache of the intermediary layer, it obtains the missing target vector data of the current layer from the storage layer through the intermediary layer and performs vector retrieval calculation with the query vector data.

[0009] Optionally, the intermediary layer further includes a second application programming interface and a third application programming interface; The computing layer updates the vector data in its own memory to the mediation layer and the storage layer for storage through the second application programming interface of the mediation layer; The computing layer hangs the target task at the tail of the event queue through the third application programming interface of the mediation layer, and waits for the mediation layer to read the missing target vector data of the current layer from the storage layer. When the mediation layer finishes reading the missing target vector data of the current layer from the storage layer, the computing layer continues to execute the target task; wherein, the target task represents obtaining the target vector data of the current layer and performing vector retrieval calculation of the current layer with the query vector data.

[0010] Optionally, when the target vector data is missing in the memory of the computing layer, the computing layer batch obtains the missing target vector data from the mediation layer and / or through the mediation layer from the storage layer for vector retrieval calculation with the query vector data, including: When the target vector data is missing in the memory of the computing layer and the data loading condition is met, the computing layer batch obtains the missing target vector data from the mediation layer and / or through the mediation layer from the storage layer for vector retrieval calculation with the query vector data, and the data loading condition is used to trigger batch loading of the missing target vector data in the memory of the computing layer itself.

[0011] Optionally, the storage layer includes a data access module, and the data access module is used to manage the data loading condition, and the data loading condition includes a first data loading condition and a second data loading condition; Wherein, the first data loading condition is: in the process of obtaining the target vector data of the current layer from its own memory according to the identifier of the current layer and performing vector retrieval calculation of the current layer with the query vector data, when the accumulated number of missing target vector data of the current layer is less than or equal to the missing threshold, suspending the loading of the missing target vector data corresponding to the current layer, and triggering batch loading after obtaining the target vector data of the current layer from its own memory according to the identifier of the current layer and performing vector retrieval calculation of the current layer with the query vector data; The second data loading condition is: in the process of obtaining the target vector data of the current layer from its own memory according to the identifier of the current layer and performing vector retrieval calculation of the current layer with the query vector data, when the accumulated number of missing target vector data of the current layer is greater than the missing threshold, triggering batch loading.

[0012] Optionally, when the intermediate layer performs the current layer vector retrieval calculation on the target vector data of the current layer obtained from its own memory according to the identifier of the current layer by the cache module in the calculation layer and the query vector data, it preloads the target vector data of the next layer.

[0013] Optionally, the intermediate layer includes a memory optimization module. When deploying the vector retrieval system for the browser environment to a device, the memory optimization module determines the memory capacity of the calculation layer according to the following heuristic algorithm: The memory optimization module performs a benchmark test with the maximum available memory capacity to obtain key parameters, where the key parameters at least include the number of accesses, the time consumption for a single query, and the total query time consumption; The memory optimization module determines an access count threshold according to the key parameters, where the access count threshold represents the number of accesses that satisfy that reducing the memory capacity will not cause the query time consumption to exceed the time consumption threshold; The memory optimization module iteratively tests to establish a quantitative relationship between the memory capacity and the number of accesses, predicts the next memory capacity test point based on linear interpolation in combination with the access count threshold, and determines the memory capacity of the calculation layer under the condition of meeting the iterative convergence condition; Among them, the memory capacity value of the calculation layer represents the minimum memory capacity under the condition of satisfying less than the time consumption threshold. The iterative convergence condition includes that the current access count test value point is greater than or equal to the access count threshold or the memory capacity is reduced to the lowest feasible capacity value, and the current access count test value point is determined according to the current memory capacity test point.

[0014] Optionally, the quantitative relationship is a linear relationship; the memory optimization module iteratively tests to establish a quantitative relationship between the memory capacity and the number of accesses, predicts the next memory capacity test point based on linear interpolation in combination with the access count threshold, and determines the memory capacity of the calculation layer, including: The memory optimization module sets the memory capacity to 0, determines the extreme value point of the number of accesses, and randomly determines a current memory capacity test point for testing to obtain the current access count test value point; The memory optimization module takes the line connecting the extreme value point of the number of accesses and the current access count test value point as the linear relationship between the current memory capacity and the number of accesses; The memory optimization module determines the next memory capacity test point according to the linear relationship between the current memory capacity and the number of accesses and the access count threshold; The memory optimization module performs a test according to the next memory capacity test point to obtain the next access count test value point; The memory optimization module repeats the above steps, iteratively tests to establish a linear relationship between the memory capacity and the access times until the current access times test value point is greater than or equal to the access times threshold, terminates the iteration, and uses the previous memory capacity test point as the memory capacity of the computing layer; or, in the case where the current memory capacity test point is reduced to the lowest feasible capacity value, terminates the iteration and uses the current memory capacity test point as the memory capacity of the computing layer.

[0015] Optionally, the storage layer includes a first key-value table and a second key-value table. The first key-value table uses the identifier of the vector data as the key and the original data corresponding to the vector data as the value; the second key-value table uses the identifier of the vector data as the key and the vector embedding corresponding to the vector data as the value. The storage layer stores all the vector data in the form of key-value pairs according to the first key-value table and the second key-value table.

[0016] The embodiments of the present application include the following advantages: In the embodiments of the present application, the vector retrieval system for the browser environment includes a task scheduling module and a hierarchical data management module. After the task scheduling module receives the query vector data of the user, it schedules the hierarchical data management module to execute the vector retrieval task. Since the hierarchical data management module adopts a three-layer data management architecture, including a computing layer storing the first part of the high-frequency access vector data, an intermediate layer storing the second part of the high-frequency access vector data, and a storage layer storing all the vector data, when the hierarchical data management module responds to the vector retrieval task, the computing layer can directly obtain the vector data from its own memory and perform vector retrieval calculation with the query vector data, without accessing the storage layer every time. Only when the target vector data is missing in its own memory, the computing layer batches and obtains the missing target vector data from the intermediate layer and / or through the intermediate layer from the storage layer to perform vector retrieval calculation with the query vector data. In this way, based on the three-layer data management architecture, the synchronous / asynchronous execution model is effectively coordinated to ensure the seamless connection between high-performance computing and browser storage, reduce the access times to the storage layer, avoid loading redundant vector data, and overcome the lack of storage access efficiency. Moreover, by adding a computing layer in the system to perform vector retrieval calculation (i.e., accelerating computationally intensive operations through the computing layer), the parsing and execution bottleneck of the existing browser using the JavaScript engine is overcome, and the vector retrieval calculation performance is improved. Therefore, based on this system, efficient vector retrieval for the browser environment can be realized. Description of the Drawings

[0017] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 is a schematic diagram of a vector retrieval system for a browser environment provided by an embodiment of the present application; Figure 2 is a schematic diagram of a heuristic memory optimization mechanism provided by an embodiment of the present application; Figure 3 is a schematic diagram of another vector retrieval system for a browser environment provided by an embodiment of the present application. Detailed implementation manners

[0019] To make the above objects, features, and advantages of the present application more obvious and understandable, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0020] In the related art, the Hierarchical Navigable Small World (HNSW) algorithm is mainly used as the index architecture for the browser-side vector retrieval engine. Although such solutions have initially realized the vector retrieval function in the browser environment, they still face at least the following technical bottlenecks in actual deployment: 1) Computational performance bottleneck. The parsing and execution characteristics of the browser JavaScript engine result in low efficiency of computationally intensive operations; a large number of vector similarity and sorting operations are required in HNSW queries. Through experimental data, the computational latency required for the existing browser-side vector retrieval engine to execute a single HNSW query exceeds 100 milliseconds, and this latency becomes a performance bottleneck for network applications and is difficult to meet the real-time interaction requirements in the browser.

[0021] 2) Storage access efficiency bottleneck: The heuristic data loading mechanism with a fixed prefetch size is adopted to load the missing data from the IndexedDB storage into the memory, resulting in prominent redundant data loading problems. Experimental data shows that when the memory size is lower than 98% of the total data size, the redundancy rate of the prefetched data exceeds 80%, leading to a large amount of invalid I / O (Input / Output) latency. At the same time, frequent IndexedDB storage accesses (such as a single retrieval triggering more than ten or even more than a hundred IndexedDB storage accesses) cause the access latency to soar to the second level, seriously affecting the user experience.

[0022] 3) Rigid utilization of memory resources: The existing vector retrieval engines on the browser side adopt a static memory allocation strategy, which cannot adapt to the dynamic resource constraints of the browser environment. The typical manifestation is that the predefined cache size cannot be automatically adjusted according to the device memory capacity, which is likely to cause memory overflow on low-end mobile devices and lead to browser crashes. This static memory allocation strategy results in serious resource waste or performance degradation problems when the system is deployed across devices.

[0023] To overcome the limitations of related technologies, the embodiments of the present application provide a vector retrieval system for the browser environment. The system adopts a three-layer data management architecture. When performing a vector retrieval task, the computing layer can directly obtain vector data from its own memory and perform vector retrieval calculations with the query vector data without accessing the storage layer every time. Only when the target vector data is missing in its own memory, the computing layer batches and obtains the missing target vector data and query vector data from the mediation layer and / or through the mediation layer from the storage layer to perform vector retrieval calculations. Therefore, it effectively coordinates the synchronous / asynchronous execution model, ensures the seamless connection between high-performance computing and browser storage, reduces the number of accesses to the storage layer, avoids loading redundant vector data, and overcomes the lack of storage access efficiency. Moreover, by performing vector retrieval calculations through the computing layer, it overcomes the parsing and execution bottleneck of the existing JavaScript engine in browsers and improves the performance of vector retrieval calculations. Therefore, efficient vector retrieval in the browser environment is achieved.

[0024] In some embodiments, only when the data loading conditions are met, the system triggers to batch obtain the missing target vector data and the query vector data from the mediation layer and / or through the mediation layer from the storage layer to perform vector retrieval calculations, dynamically adjusting the vector data loading timing to further avoid redundant data prefetching.

[0025] In some embodiments, when deploying the vector retrieval system for the browser environment to a device, the memory capacity of the computing layer is determined through a heuristic algorithm, so that the memory capacity can be dynamically adjusted according to the device performance, saving memory occupancy while ensuring the query latency.

[0026] The vector retrieval system for browser environment provided by the embodiments of the present application will be described below with reference to the accompanying drawings.

[0027] Refer to Figure 1 as shown Figure 1 is a schematic diagram of a vector retrieval system for browser environment provided by the embodiments of the present application. As Figure 1 shown, the vector retrieval system for browser environment includes a task scheduling module and a hierarchical data management module, where: The task scheduling module receives the query vector data of the user and schedules the hierarchical data management module to execute the vector retrieval task, and the query vector data represents the query information provided by the user; The hierarchical data management module includes a computing layer storing the first part of the frequently accessed vector data, an intermediate layer storing the second part of the frequently accessed vector data, and a storage layer storing all vector data, where the frequently accessed vector data represents the vector data whose access times in the target time period are greater than the access times threshold; In response to the vector retrieval task, the computing layer in the hierarchical data management module obtains the target vector data from its own memory to perform vector retrieval calculation with the query vector data, and when the target vector data is missing in its own memory, the computing layer batches to obtain the missing target vector data from the intermediate layer and / or through the intermediate layer from the storage layer to perform vector retrieval calculation with the query vector data.

[0028] In the embodiments of the present application, the computing layer is responsible for performing efficient vector retrieval calculations on the browser side (for example, HNSW graph traversal, vector similarity calculation, candidate set sorting, etc.). The computing layer is compiled from the core computing logic of the HNSW algorithm (for example, vector similarity calculation, candidate set sorting, etc.) (for example, compiled using the Emscripten toolchain, where Emscripten is an open-source compiler toolchain). Compared with using a JavaScript engine, the computing layer can achieve performance close to native code. In some embodiments, the computing layer is implemented based on WebAssembly. Among them, WebAssembly (i.e., Wasm) is a code compilation format that enables programs written in high-level languages to run in web browsers and has the characteristic of near-native execution speed. The intermediate layer uses a JavaScript intermediate layer, which serves as a bridge between the computing layer and the storage layer, for complex data serialization, asynchronous task scheduling, and memory monitoring. The storage layer refers to the IndexedDB storage layer, which is responsible for storing the original vector data, that is, storing all vector data.

[0029] To solve the problems of limited computing performance, insufficient memory capacity, and low storage access efficiency in the browser environment, the system constructs a three-level data management architecture (i.e., the hierarchical data management module). By caching high-frequency accessed vector data (i.e., the first part of high-frequency accessed vector data) in the memory of the computing layer, and using the mediation layer as an intermediate cache to manage the additional vector data (i.e., the second part of high-frequency accessed vector data) that cannot be cached by the computing layer; IndexedDB is used as persistent storage (storage layer) to store the complete index structure and vector data, that is, to store all vector data, where all vector data includes the first part of high-frequency accessed vector data, the second part of high-frequency accessed vector data, and the remaining vector data. Through the three-level data management architecture, the problems of limited computing performance, insufficient memory capacity, and low storage access efficiency in the browser environment are solved.

[0030] The task scheduling module is responsible for managing the hierarchical data management module. During use, for the query vector data of the user (i.e., the input / output of the user), the task scheduling module schedules the computing layer to continuously execute high-density vector retrieval calculations; among them, performing vector retrieval calculations on the target vector data and the query vector data can be understood as: calculating the similarity between the target vector data and the query vector data, and sorting according to the similarity calculation results, so as to filter out the vector data that meets the user's needs according to the sorting.

[0031] Specifically, since the first part of high-frequency accessed vector data is stored in the computing layer, the computing layer first obtains the corresponding target vector data from its own memory and performs vector retrieval calculations with the query vector data. If all target vector data can be obtained from its own memory for vector retrieval calculations with the query vector data, there is no need to access the vector data in the mediation layer and the storage layer; if the target vector data is missing in the computing layer's own memory (i.e., the corresponding target vector data cannot be obtained in its own memory), the missing target vector data is obtained from the mediation layer or the storage layer in batches and vector retrieval calculations are performed with the query vector data; obtaining the missing target vector data in batches reduces the number of accesses to the storage layer and avoids loading redundant vector data; moreover, the vector retrieval calculations in the computing layer and the storage access in the storage layer are executed asynchronously, maximizing the utilization of computing resources and I / O bandwidth.

[0032] It is understandable that in this system, the vector data in the computing layer, the mediation layer, and the storage layer are all stored separately in text-vector form, and the corresponding data is retrieved through a unique identifier (Identifier, ID), thereby reducing the storage overhead. Moreover, for large index files (i.e., files whose size exceeds the threshold), indexing is performed in a sharded manner. In addition, the system adopts a pluggable cache eviction policy (such as the default FIFO (First In First Out)), which balances the access efficiency of hot and cold data. These optimizations work in conjunction with the core algorithm to achieve efficient retrieval of vectors.

[0033] Through the above implementation process, the system adopts a three-layer data management architecture, effectively coordinates the synchronous / asynchronous execution model, ensures seamless connection between high-performance computing and browser storage, reduces the number of accesses to the storage layer, avoids loading redundant data, and overcomes the lack of storage access efficiency. Moreover, by adding a computing layer in this system for vector retrieval calculation (i.e., accelerating compute-intensive operations through the computing layer), it overcomes the parsing and execution bottleneck of the existing browser using the JavaScript engine and improves the performance of vector retrieval calculation. Therefore, based on this system, efficient vector retrieval in the browser environment can be achieved.

[0034] The vector retrieval system for the browser environment provided by this application will be described in detail below through Section 1.1 Three-Layer Data Management Architecture, Section 1.2 Phased Lazy Loading Mechanism, and Section 1.3 Heuristic Memory Optimization Mechanism respectively.

[0035] 1.1 Three-Layer Data Management Architecture: In an alternative embodiment, the computing layer is implemented based on WebAssembly. The computing layer includes a hierarchical index structure and a cache module. Each layer of the hierarchical index structure is composed of the identifiers of the vector data. Specifically, the computing layer obtains the target vector data from its own memory for vector retrieval calculation with the query vector data, including: the cache module of the computing layer sequentially obtains the target vector data of the current layer from its own memory according to the identifiers of the current layer in the hierarchical index structure, and performs vector retrieval calculation of the current layer with the query vector data.

[0036] In the embodiments of the present application, the computing layer is implemented based on WebAssembly. Since WebAssembly is limited by 32-bit addressing and cannot directly access IndexedDB, the vector data in the storage layer can be obtained through the three-level data management interface of the hierarchical data management module via the mediation layer. The computing layer includes a cache module, which is used to maintain a cross-layer cache. The cache module uses the identifier of the vector data (i.e., the ID of the vector data) as the key to cache each group of loaded vector data. The computing layer organizes all vector data through a hierarchical index structure, which includes multiple layers, and each layer is composed of the identifiers of the vector data. The computing layer performs vector retrieval calculations layer by layer.

[0037] It can be understood that in the hierarchical index structure, the earlier layer (for example, the first layer) usually corresponds to the identifiers of the frequently accessed vectors. Therefore, when performing vector retrieval according to the hierarchical index structure, the corresponding target vector data can be preferentially obtained from the calculated memory and used for vector retrieval calculation with the query vector data. Moreover, the hierarchical index structure can be adjusted, and the computing layer can adjust the identifiers of each layer in the hierarchical index structure according to the access frequency of the vector data.

[0038] When the hierarchical data management module responds to a vector retrieval task, the computing layer, through the cache module and according to the hierarchical index structure, sequentially obtains the target vector data of the current layer from its own memory based on the identifier of the current layer, and performs vector retrieval calculation of the current layer with the query vector data. If the computing layer lacks the target vector data of the current layer in its own memory, it batches and obtains the missing target vector data of the current layer from the mediation layer and / or through the mediation layer from the storage layer.

[0039] Furthermore, the mediation layer includes a first application programming interface; when the computing layer lacks the target vector data in its own memory, batch obtaining the missing target vector data from the mediation layer and / or through the mediation layer from the storage layer for vector retrieval calculation with the query vector data includes: Step A1: When the computing layer lacks the target vector data of the current layer in its own memory, it notifies the mediation layer to check the heap cache through the first application programming interface of the mediation layer, and obtains the missing target vector data of the current layer from the heap cache of the mediation layer for vector retrieval calculation with the query vector data; Step A2: When the heap cache of the mediation layer does not have the missing target vector data of the current layer, it obtains the missing target vector data of the current layer from the storage layer through the mediation layer for vector retrieval calculation with the query vector data.

[0040] In the embodiments of the present application, when the computing layer performs vector retrieval calculation, it obtains the corresponding target vector data in strict data access order, that is, it preferentially obtains the target vector data from its own memory, then checks the heap cache of the mediation layer, and finally triggers the access to the storage layer.

[0041] Specifically, for each level in the corresponding hierarchical index structure, the target vector data of the current level is preferentially obtained from the memory of the computing layer and vector retrieval calculation is performed with the query vector data; if the target vector data of the current level is missing in the memory of the computing layer itself, then step A1 is executed, and the mediation layer is notified through the first application programming interface (that is, the get() API, where API refers to the Application Programming Interface) to check the push cache to obtain the missing target vector data of the current level and perform vector retrieval calculation with the query vector data. If all the missing target vector data of the current level can be obtained from the heap cache of the mediation layer, the vector retrieval of the current level can be ended. If there are still missing target vector data, then step A2 is continued to obtain the missing target vector data of the current level from the storage layer and perform vector retrieval calculation with the query vector data. The mediation layer serves as a bridge between the computing layer and the storage layer, and when the computing layer needs to access the storage layer, it accesses the storage layer through the mediation layer.

[0042] Through the above implementation process, vector data is stored based on a three-layer data management architecture, and the target vector data and query vector data are retrieved and calculated according to the data access order. Through this multi-level cache mechanism, the external storage access overhead is significantly reduced.

[0043] In an alternative embodiment, during the process that the mediation layer obtains the target vector data of the current level from its own memory according to the identifier of the current level through the cache module of the computing layer and performs the current level vector retrieval calculation with the query vector data, the target vector data of the next level is pre-loaded.

[0044] Specifically, during the process that the task scheduling module schedules the computing layer to perform the vector retrieval of the current level, the target vector data of the next level is asynchronously pre-loaded through the mediation layer at the same time (for example, the target vector data of the next level is loaded from the storage layer). In this way, the acquisition efficiency of vector data is improved, and efficient vector retrieval in the browser environment is realized.

[0045] In an alternative embodiment, the mediation layer further includes a second application programming interface and a third application programming interface; The computing layer updates the vector data in its own memory to the mediation layer and the storage layer for storage through the second application programming interface of the mediation layer; The computing layer suspends the target task at the tail of the event queue through the third application programming interface of the mediation layer, waiting for the mediation layer to read the missing target vector data of the current layer from its own heap cache and / or from the storage layer. When the mediation layer finishes reading the missing target vector data of the current layer from its own heap cache and / or through the mediation layer from the storage layer, the computing layer continues to execute the target task; wherein, the target task represents obtaining the target vector data of the current layer and performing vector retrieval calculation of the current layer with the query vector data.

[0046] In the embodiment of the present application, as a bridge between the computing layer and the storage layer, the mediation layer can also update the vector data in the memory of the computing layer to the storage layer for storage through the second application programming interface of the mediation layer (i.e., the store() API). In this way, through the data transfer function implemented by the mediation layer through the above-mentioned first application programming interface and second application programming interface, that is, the vector data in the storage layer can be obtained for the computing layer through the first application programming interface, and the vector data of the computing layer can be stored in the middle storage layer through the second application programming interface.

[0047] The mediation layer can also perform a model coordination function, that is, the synchronous computing model of the computing layer and the asynchronous I / O of the storage layer are coordinated through the mediation layer event loop. Specifically, when the computing layer is waiting for the mediation layer to obtain the missing target vector data of the current layer from the storage layer, it suspends the target task at the tail of the event queue through the third application programming interface (i.e., the wait() API), and after the mediation layer finishes reading the missing target vector data of the current layer from the storage layer, it notifies the computing layer to resume executing the target task through a shared signal.

[0048] Through the above implementation process, through the application programming interfaces deployed in the mediation layer (for example, the first program programming interface, the second program programming interface, the third program programming interface), the mediation layer acts as a bridge between the synchronous and asynchronous execution models, and coordinates the synchronous vector retrieval calculation of the computing layer and the asynchronous I / O operation of the computing layer through the shared signal mechanism; when a cache miss occurs in the computing layer (that is, the memory of the computing layer lacks target vector data), the mediation layer is responsible for batch loading data from the storage layer and ensuring the efficient execution of computing tasks through event loop scheduling.

[0049] In an alternative embodiment, the storage layer includes a first key-value table and a second key-value table. The first key-value table uses the identifier of the vector data as the key and the original data corresponding to the vector data as the value; the second key-value table uses the identifier of the vector data as the key and the vector embedding corresponding to the vector data as the value; The storage layer stores all the vector data in the form of key-value pairs according to the first key-value table and the second key-value table.

[0050] In the embodiments of the present application, for all vector data, the original data corresponding to the vector data is stored as a value in the first key-value table, and the original data corresponding to the vector data is stored as a value in the second key-value table. By this separated storage method, data retrieval and data transmission in the storage layer can be accelerated.

[0051] 1.2 Phased lazy loading mechanism: In an optional embodiment, when the computing layer lacks target vector data in its own memory, it batch retrieves the missing target vector data from the mediation layer and / or through the mediation layer from the storage layer to perform vector retrieval calculation with the query vector data, including: When the computing layer lacks target vector data in its own memory and meets the data loading condition, it batch retrieves the missing target vector data from the mediation layer and / or through the mediation layer from the storage layer to perform vector retrieval calculation with the query vector data. The data loading condition is used to trigger batch loading of the missing target vector data in the computing layer's own memory.

[0052] In the embodiments of the present application, considering that the traditional prefetching strategy will cause a large amount of redundant data loading when the memory is limited, resulting in a decline in the vector retrieval performance of the system. Therefore, the embodiments of the present application propose a phased lazy loading (PhasedLazy Loading) mechanism to dynamically adjust the data loading timing. Specifically, during the process of performing vector retrieval calculation in the computing layer, if the target vector data is missing in its own memory, it will not immediately retrieve the missing target vector data from the mediation layer or the storage layer to perform vector retrieval calculation with the query vector data. Instead, it will trigger batch loading of the missing target vector data in the computing layer's own memory only when the data loading condition is met, thereby improving the storage access efficiency.

[0053] Furthermore, the storage layer includes a data access module, and the data access module is used to manage the data loading condition. The data loading condition includes a first data loading condition and a second data loading condition; Among them, the first data loading condition is: during the process of retrieving the target vector data of the current layer from its own memory according to the identifier of the current layer and performing current layer vector retrieval calculation with the query vector data, when the cumulative number of missing target vector data of the current layer is less than or equal to the missing threshold, loading of the missing target vector data corresponding to the current layer is postponed, and after retrieving the target vector data of the current layer from its own memory according to the identifier of the current layer and performing current layer vector retrieval calculation with the query vector data, batch loading is triggered; The second data loading condition is as follows: when obtaining the target vector data of the current layer from its own memory according to the identifier of the current layer and performing the current layer vector retrieval calculation with the query vector data, if the number of the accumulated missing target vector data of the current layer is greater than the missing threshold, batch loading is triggered.

[0054] In the embodiments of the present application, during the vector retrieval process, when the calculation layer lacks the target vector data in its own memory, batch loading will only be triggered when the above first data loading condition or the second data loading condition is met. Herein, batch loading refers to batch obtaining the missing target vector data from the intermediate layer and / or from the storage layer through the intermediate layer.

[0055] Specifically, for the first data loading condition, it can also be understood as the inter-layer loading stage; during the process of performing the current layer vector retrieval calculation, as long as the number of the accumulated missing target vector data of the current layer is less than or equal to the missing threshold, the missing target vector data is temporarily ignored, and after the vector retrieval calculation of the current layer is completed in the memory, all the missing target vector data of this layer is uniformly loaded to ensure the correct entry point of the next layer. For example, during the process of performing the current layer vector retrieval calculation, if the target vector data to be accessed by the current layer is , where misses the memory (memory missing), then the loading of is postponed, and the in the memory is continued to complete the vector retrieval calculation (search) of the current layer, and all and its neighbors are batch loaded when the vector retrieval calculation of this layer ends. In this way, by triggering batch loading through the first data loading condition, a large amount of redundant loading can be reduced, and at the same time, the correct entry point of the next layer is ensured.

[0056] For the second data loading condition, it can also be understood as the intra-layer loading stage; during the process of performing the current layer vector retrieval calculation, as long as the number of the accumulated missing target vector data of the current layer is greater than the missing threshold, batch loading is immediately triggered to avoid deviation of the query path. For example, the missing threshold is 100. If 201 unhit target vector data are accumulated during the vector retrieval calculation of a certain layer, the vector retrieval calculation is interrupted, and all the missing target vector data is directly obtained from the intermediate layer and / or from the storage layer through the intermediate layer to avoid result deviation caused by the breakage of the search path.

[0057] Through the above implementation process, when the calculation layer lacks the target vector data in its own memory, batch loading of the missing target vector data is triggered through the data loading conditions (i.e., the first data loading condition and the second data loading condition). In this way, while ensuring the query accuracy, the access frequency to the storage layer is reduced.

[0058] 1.3 Heuristic Memory Optimization Mechanism: In an alternative embodiment, the mediation layer includes a memory optimization module. When deploying the vector retrieval system for the browser environment to a device, the memory optimization module determines the memory capacity of the computing layer according to the following heuristic algorithm: Step B1: The memory optimization module benchmarks with the maximum available memory capacity to obtain key parameters, where the key parameters at least include the number of accesses, the time consumed for a single query, and the total query time. Step B2: The memory optimization module determines an access count threshold according to the key parameters, where the access count threshold represents the number of accesses that satisfy that reducing the memory capacity will not cause the query time to exceed the time threshold. Step B3: The memory optimization module iteratively tests to establish a quantitative relationship between the memory capacity and the number of accesses, predicts the next memory capacity test point based on linear interpolation in combination with the access count threshold, and determines the memory capacity of the computing layer under the condition of meeting the iterative convergence condition. Among them, the memory capacity value of the computing layer represents the minimum memory capacity under the condition of satisfying being less than the time threshold. The iterative convergence condition includes that the current access count test value point is greater than or equal to the access count threshold or the memory capacity is reduced to the lowest feasible capacity value, and the current access count test value point is determined according to the current memory capacity test point.

[0059] In the embodiment of the present application, the optimal memory threshold is adaptively determined through iterative testing and mathematical model calculation to minimize memory occupancy while ensuring query performance. Specifically, first, the initial test phase of Step B1 is executed, and benchmark testing is performed with the maximum available memory capacity (for example, the 256MB memory capacity allowed by the browser environment) to obtain key parameters. Among them, the key parameters are used for subsequent calculation of the optimal memory capacity to ensure that the system will not cause a sudden drop in query performance due to excessive compression of the cache while reducing memory occupancy; the key parameters at least include the number of accesses, the time consumed for a single query, and the total query time. The number of accesses refers to the number of times of accessing the storage layer during a single query (vector retrieval calculation). The time consumed for a single query refers to the average time for loading data from the storage layer, and the total query time refers to the complete time from the start of the query to the return of the result. In addition, the key parameters can also include the query path length, that is, the number of vector data traversed.

[0060] Next, perform the iterative optimization phase of steps B2 and B3. Based on the initial test data, the system dynamically adjusts the memory size using a mathematical model and linear interpolation method until the optimal memory capacity is found. In step B2, the memory optimization module determines the access count threshold according to the key parameters. One access count can be determined by the proportional threshold method and the other by the absolute delay threshold method, and then the larger one is used as the access count threshold to ensure that the query time consumption is always controllable. For example, if the access count is 3 times, the single query time consumption is 32 milliseconds, and the total query time consumption is 186 milliseconds, the process of determining the access count by the proportional threshold method is as follows: Set the proportion of the access time consumption not exceeding the total query time consumption (such as 0.8), then the access count is , that is, the system allows at most 4 to 5 accesses; the process of determining the access count by the delay threshold method is as follows: Set the access time consumption not exceeding the target query time consumption (the target query time consumption is less than the total query time consumption, such as the target query time consumption is 100 milliseconds), then the access count is , that is, the system allows at most 3 accesses; finally, the access count threshold of 5 times can be determined.

[0061] In step B3, a heuristic method is used to dynamically determine the optimal memory capacity of the computing layer, avoiding resource waste or performance degradation caused by static allocation. Its core idea is to model the query process as a black box system, establish a quantitative relationship between the memory capacity and the access count (representing the query time consumption) through iterative testing, and intelligently adjust the memory capacity allocation strategy based on the quantitative relationship. Specifically, starting from the maximum memory capacity, gradually reduce the memory occupancy ratio and monitor the access count of the storage layer, and approximate the optimal memory point through linear interpolation to ensure that the access time consumption does not exceed the time consumption threshold (for example, 20% of the total access time consumption or an absolute value of 100 ms). In this way, the system can adapt to different device performances, save an average of 31% of the memory occupancy in the browser, and at the same time keep the query latency stable.

[0062] Furthermore, the quantitative relationship is a linear relationship, and step B3 may include sub-steps B31 to B35: Step B31: The memory optimization module sets the memory capacity to 0, determines the extreme point of the access count, and randomly determines a current memory capacity test point for testing to obtain the current access count test value point; Among them, when the memory capacity is 0, theoretically each data access requires external data access, that is, the computing layer retrieves and calculates vectors for each layer and obtains the corresponding target vector data from the mediation layer or the storage layer. The current access count test value refers to the actual access count at the current memory capacity test point.

[0063] Step B32: The memory optimization module uses the line connecting the extreme point of the access count and the current access count test value point as the linear relationship between the current memory capacity and the access count; Exemplarily, the linear relationship between the current memory capacity (C) and the number of accesses ( ) can be expressed as: , where the slope k reflects the degree of influence of the change in memory capacity on the number of accesses, and the intercept b is related to the deployed device, and the value of b corresponding to different devices is different.

[0064] Step B33: The memory optimization module determines the next memory capacity test point according to the linear relationship between the current memory capacity and the number of accesses, and the access number threshold; Specifically, the access number threshold can be substituted into the linear relationship between the current memory capacity and the number of accesses, and the next memory capacity test point can be obtained, that is, the memory capacity at the intersection of the line connecting the access number extreme point and the current access number test value point and the line where y is equal to the access number threshold is used as the next memory capacity test point.

[0065] Step B34: The memory optimization module performs a test according to the next memory capacity test point to obtain the next access number test value point; Step B35: The memory optimization module repeats the above steps, iteratively tests to establish the linear relationship between the memory capacity and the number of accesses until the current access number test value point is greater than or equal to the access number threshold, ends the iteration, and uses the previous memory capacity test point as the memory capacity of the computing layer; or, when the current memory capacity test point is reduced to the lowest feasible capacity value, ends the iteration, and uses the current memory capacity test point as the memory capacity of the computing layer.

[0066] Specifically, if the current access number test value point is less than the access number threshold, it means that reducing the memory at this time will not cause the query time to exceed the threshold, that is, continue to iteratively test to establish the linear relationship between the memory capacity and the number of accesses. If the current access number test value point is greater than or equal to the access number threshold, it means that reducing the memory at this time will affect the query time (query latency), and no longer iterate, so roll back to the previous capacity test point, and use the previous memory capacity test point as the memory capacity of the computing layer. In this way, through the memory rollback mechanism, when the current access number test value point is greater than or equal to the access number threshold, it automatically restores to the previous level of safe memory value to ensure service reliability. And when the current memory capacity test point is reduced to the lowest feasible capacity value, it means that the optimal memory capacity has been found, and at this time, the current memory capacity test point is used as the content capacity of the computing layer.

[0067] As Figure 2 shown, Figure 2 is a schematic diagram of a heuristic memory optimization mechanism provided by an embodiment of the present application, Figure 2It shows the process of iteratively testing to establish the quantitative relationship between memory capacity and access times, and predicting the next memory capacity test point based on linear interpolation combined with the access time threshold; among them, the solid line represents the best quantitative relationship between memory capacity and access times, and the dashed line and dotted line represent the linear relationship between memory capacity and access times, where represents the maximum access times, and N represents the maximum memory capacity.

[0068] Specifically, first, based on the maximum access times establish the extreme point A of access times, and the random content capacity establish the current access time test value point , and take the line connecting the extreme point A of access times and the current access time test value point as the linear relationship between the current memory capacity and access times; if the current access time test value point is less than the access time threshold , obtain the memory capacity at the intersection of the line and as the next memory capacity test point; according to determine the next access time test value point , and take the line connecting the extreme point A of access times and the next access time test value point as the linear relationship between the next memory capacity and access times, obtain the memory capacity at the intersection of the line and as the memory capacity at the intersection as the next-next memory capacity test point. In this way, repeat the above steps until the iterative convergence condition is met to determine the memory capacity of the computing layer. By the above implementation process, a heuristic method is used to dynamically determine the optimal memory capacity of the computing layer, avoiding resource waste or performance degradation caused by static allocation. Therefore, the system can automatically adjust the memory occupancy according to device performance, browser type, and dataset size; through a mathematical model, it ensures that the query time is always controllable and avoids a sharp drop in performance caused by memory compression. Compared with the fixed memory allocation strategy of the prior art, it can save 7% - 39% of memory and improve the compatibility of low-end devices. In this way, it ensures that the system can maintain efficient and stable retrieval performance in different hardware environments, and solves the problem of resource waste or performance fluctuations caused by static memory allocation in traditional solutions.

[0069] As

[0070] shown Figure 3 in Figure 3It is a schematic diagram of another vector retrieval system for the browser environment provided by an embodiment of the present application. The system includes a task scheduling module and a hierarchical data management module. The hierarchical data management module is a three-layer data management architecture, including a computing layer, an intermediary layer, and a storage layer. Among them, the computing layer contains a cache module, and the cache module is used to maintain a cross-layer cache; the intermediary layer contains a memory optimization module. When the system is deployed to a device, the memory optimization module adjusts the content capacity of the computing layer through a heuristic algorithm and detects the query latency (access time). Under the premise of meeting the preset latency threshold, it uses as little memory as possible; the storage layer contains a data access module, and the data access module uses lazy loading technology to manage the data loading conditions to reduce the storage access frequency of the storage layer.

[0071] Specifically, after receiving the query vector data of the user, the task scheduling module schedules the hierarchical data management module to execute the vector retrieval task. In response to the vector retrieval task, the hierarchical data management module first uses the cache module of the computing layer to sequentially obtain the target vector data of the current layer from its own memory according to the identifier of the current layer according to the hierarchical index structure, and performs the vector retrieval calculation of the current layer with the query vector data. When the computing layer lacks the target vector data in its own memory and meets the data loading conditions, it batch obtains the missing target vector data from the intermediary layer and / or through the intermediary layer from the storage layer to perform the vector retrieval calculation with the query vector data, so as to achieve efficient vector retrieval in the browser environment.

[0072] Based on the vector retrieval system for the browser environment provided by the embodiment of the present application, compared with the prior art, it has the following significant advantages: Advantage 1: The computing performance is greatly improved. The system accelerates computationally intensive operations (such as vector similarity calculation, sorting, etc.) through the computing layer (WebAssembly), overcoming the parsing and execution bottleneck of the traditional JavaScript engine. Experimental data shows that in the same hardware environment, the P99 (P99 refers to the performance metric that measures that 99% of the requests are within the maximum latency time in the service response time distribution) query latency of the system is reduced by 70 to 743 times, optimized from the second level to the millisecond level, meeting the real-time interaction requirements.

[0073] Advantage 2: The storage access efficiency is optimized. Adopting the lazy loading mechanism (Phased Lazy Loading), dynamically adjusts the data loading timing, and avoids redundant data prefetching. Compared with the existing fixed prefetching mechanism, when the memory coverage rate is lower than 98%, the system reduces more than 50% of the invalid disk I / O, enabling the retrieval of large-scale data sets (such as 7.5GB Wiki-480k) to still maintain a response at the 10ms level.

[0074] Advantage 3: Improved memory resource utilization. Based on the heuristic memory optimization mechanism, the system can dynamically adjust the cache threshold according to the device performance, saving 7% - 39% of the memory occupancy while ensuring the query latency (e.g., <100ms). In addition, the text-vector separation storage design further reduces the memory overhead per entry, enabling the system to still operate stably on low-end devices.

[0075] Advantage 4: Enhanced cross-device compatibility. Through a three-tier data management architecture based on the hierarchical data management module, it effectively coordinates the synchronous / asynchronous execution model to ensure seamless connection between high-performance computing and browser storage. Experiments have proved that the system maintains stable performance on Chrome, Firefox, Safari (Chrome, Firefox, and Safari refer to three different browsers) and different hardware (personal computers / mobile devices), supporting the dataset scale to expand by more than 8.5 times.

[0076] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other.

[0077] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0078] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide for implementing in the processFigure 1 one process or multiple processes and / or blocks Figure 1 steps of functions specified in one block or multiple blocks.

[0080] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0081] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0082] The above has introduced in detail a vector retrieval system for a browser environment provided by the present application. Specific examples are used in this text to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A vector retrieval system for a browser environment, characterized in that It includes a task scheduling module and a hierarchical data management module, where: The task scheduling module receives the query vector data of the user and schedules the hierarchical data management module to execute the vector retrieval task, and the query vector data represents the query information provided by the user; The hierarchical data management module includes a computing layer storing the first part of the high-frequency access vector data, an intermediary layer storing the second part of the high-frequency access vector data, and a storage layer storing all vector data. Among them, the high-frequency access vector data represents the vector data whose access times in the target time period are greater than the access times threshold; In response to the vector retrieval task, the computing layer in the hierarchical data management module obtains the target vector data from its own memory to perform vector retrieval calculation with the query vector data, and when the target vector data is missing in its own memory, the computing layer batches and obtains the missing target vector data from the intermediary layer and / or through the intermediary layer from the storage layer to perform vector retrieval calculation with the query vector data.

2. The system according to claim 1, wherein The computing layer is implemented based on WebAssembly. The computing layer includes a hierarchical index structure and a cache module, and each layer of the hierarchical index structure is composed of the identifiers of vector data; The computing layer obtains the target vector data from its own memory to perform vector retrieval calculation with the query vector data, including: The cache module of the computing layer sequentially obtains the target vector data of the current layer from its own memory according to the identifiers of the current layer in accordance with the hierarchical index structure, and performs vector retrieval calculation of the current layer with the query vector data.

3. The system according to claim 2, wherein The intermediary layer includes a first application programming interface; when the target vector data is missing in its own memory, the computing layer batches and obtains the missing target vector data from the intermediary layer and / or through the intermediary layer from the storage layer to perform vector retrieval calculation with the query vector data, including: When the target vector data of the current layer is missing in its own memory, the computing layer notifies the intermediary layer to check the heap cache through the first application programming interface of the intermediary layer, obtains the missing target vector data of the current layer from the heap cache of the intermediary layer, and performs vector retrieval calculation with the query vector data; When the missing target vector data of the current layer does not exist in the heap cache of the intermediary layer, the missing target vector data of the current layer is obtained from the storage layer through the intermediary layer, and vector retrieval calculation is performed with the query vector data.

4. The system according to claim 3, characterized in that The intermediary layer also includes a second application programming interface and a third application programming interface; The computing layer updates the vector data in its own memory to the intermediary layer and the storage layer for storage through the second application programming interface of the intermediary layer; The computing layer hangs the target task at the end of the event queue through the third application programming interface of the mediation layer, and waits for the mediation layer to read the missing target vector data of the current layer from the storage layer. When the mediation layer finishes reading the missing target vector data of the current layer from the storage layer, the computing layer continues to execute the target task; wherein, the target task is to obtain the target vector data of the current layer and perform vector retrieval calculation of the current layer with the query vector data.

5. The system according to claim 2, wherein When the computing layer lacks target vector data in its own memory, it batch obtains the missing target vector data from the mediation layer and / or through the mediation layer from the storage layer for vector retrieval calculation with the query vector data, including: When the computing layer lacks target vector data in its own memory and meets the data loading condition, it batch obtains the missing target vector data from the mediation layer and / or through the mediation layer from the storage layer for vector retrieval calculation with the query vector data. The data loading condition is used to trigger the batch loading of the missing target vector data in the computing layer's own memory.

6. The system according to claim 5, wherein The storage layer includes a data access module, and the data access module is used to manage the data loading condition. The data loading condition includes a first data loading condition and a second data loading condition; Wherein, the first data loading condition is: in the process of obtaining the target vector data of the current layer from its own memory according to the identifier of the current layer and performing vector retrieval calculation of the current layer with the query vector data, when the cumulative number of missing target vector data of the current layer is less than or equal to the missing threshold, the loading of the missing target vector data corresponding to the current layer is postponed, and after obtaining the target vector data of the current layer from its own memory according to the identifier of the current layer and performing vector retrieval calculation of the current layer with the query vector data, batch loading is triggered; The second data loading condition is: in the process of obtaining the target vector data of the current layer from its own memory according to the identifier of the current layer and performing vector retrieval calculation of the current layer with the query vector data, when the cumulative number of missing target vector data of the current layer is greater than the missing threshold, batch loading is triggered.

7. The system according to any one of claims 2-6, characterized in that, The mediation layer preloads the target vector data of the next layer in the process of the computing layer obtaining the target vector data of the current layer from its own memory according to the identifier of the current layer through the cache module and performing vector retrieval calculation of the current layer with the query vector data.

8. The system according to any one of claims 1-6, characterized in that The mediation layer includes a memory optimization module. When deploying the vector retrieval system for the browser environment to a device, the memory optimization module determines the memory capacity of the computing layer according to the following heuristic algorithm: The memory optimization module performs a benchmark test with the maximum available memory capacity to obtain key parameters, and the key parameters at least include the number of accesses, the time consumption of a single query, and the total query time consumption; The memory optimization module determines an access count threshold according to the key parameter, where the access count threshold represents the access count that satisfies that reducing the memory capacity will not cause the query time to exceed the time consumption threshold; The memory optimization module iteratively tests to establish a quantitative relationship between the memory capacity and the access count, predicts the next memory capacity test point based on linear interpolation in combination with the access count threshold, and determines the memory capacity of the computing layer under the condition of meeting the iterative convergence condition; Among them, the memory capacity value of the computing layer represents the minimum memory capacity under the condition of satisfying being less than the time consumption threshold, and the iterative convergence condition includes that the current access count test value point is greater than or equal to the access count threshold or the memory capacity is reduced to the lowest feasible capacity value, and the current access count test value point is determined according to the current memory capacity test point.

9. The system according to claim 8, wherein The quantitative relationship is a linear relationship; the memory optimization module iteratively tests to establish a quantitative relationship between the memory capacity and the access count, predicts the next memory capacity test point based on linear interpolation in combination with the access count threshold, and determines the memory capacity of the computing layer, including: The memory optimization module sets the memory capacity to 0, determines the access count extreme point, and randomly determines a current memory capacity test point for testing to obtain the current access count test value point; The memory optimization module takes the connection line between the access count extreme point and the current access count test value point as the linear relationship between the current memory capacity and the access count; The memory optimization module determines the next memory capacity test point according to the linear relationship between the current memory capacity and the access count and the access count threshold; The memory optimization module tests according to the next memory capacity test point to obtain the next access count test value point; The memory optimization module repeats the above steps, iteratively tests to establish a linear relationship between the memory capacity and the access count, until the current access count test value point is greater than or equal to the access count threshold, ends the iteration, and takes the previous memory capacity test point as the memory capacity of the computing layer; or, in the case that the current memory capacity test point is reduced to the lowest feasible capacity value, ends the iteration, and takes the current memory capacity test point as the memory capacity of the computing layer.

10. The system according to claim 1, wherein The storage layer includes a first key-value table and a second key-value table. The first key-value table uses the identifier of the vector data as the key and the original data corresponding to the vector data as the value; the second key-value table uses the identifier of the vector data as the key and the vector embedding corresponding to the vector data as the value; The storage layer stores all the vector data in the form of key-value pairs according to the first key-value table and the second key-value table.

Images