An IDE Agent Skills high-precision retrieval system based on a three-layer hybrid architecture

Abstract

This application relates to the field of artificial intelligence, specifically disclosing a high-precision retrieval system for IDE Agent Skills based on a three-layer hybrid architecture, aiming to solve the problems of insufficient retrieval accuracy and excessive resource consumption in existing technologies. The system includes: a first-layer metadata inverted index module, used to build a lightweight inverted index, reducing the entire candidate set to a preset range through field-weighted BM25 scores; a second-layer lightweight semantic vector retrieval module, used to perform approximate nearest neighbor retrieval on candidate Skills and calculate cosine similarity; and a third-layer comprehensive score fusion module, used to perform weighted linear fusion of the two scores to generate the final ranking result. This application achieves overall optimization of retrieval accuracy and response speed through a three-layer progressive hybrid retrieval architecture, while all data is stored locally on the client, eliminating the risk of data leakage by eliminating the need to upload to the cloud, and enabling normal operation in an offline environment.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a high-precision retrieval system for IDE AgentSkills based on a three-layer hybrid architecture. Background Technology

[0002] With the widespread adoption of AI-assisted programming tools, the AgentSkills skill library has become a core support for the modular development of common components in modern software development. In complex business scenarios such as smart highways, the skill library can easily reach hundreds or even thousands of items. How to achieve efficient and accurate retrieval of skills directly determines development efficiency. However, existing retrieval technologies face an irreconcilable contradiction under local resource constraints: pure keyword retrieval, while fast and resource-efficient, cannot identify semantically equivalent but differently expressed skills, resulting in severely insufficient retrieval accuracy; while pure semantic vector full-scale retrieval can understand semantic meaning, when the number of skills expands to thousands, memory consumption and inference latency increase linearly, far exceeding the acceptable range for IDE plugins.

[0003] In practical applications, this contradiction manifests as follows: when developers input "database pagination query," pure keyword matching can only match skills containing the literal words "pagination" and "query," while skills with equivalent functionality but described as "data list segmented loading" or "large-scale record set offset retrieval" are missed, leading to a significant drop in the MRR (Mean Responsiveness Rate). Simultaneously, as the number of skills increases from hundreds to thousands, the memory usage of full vector retrieval increases dramatically, and retrieval latency also rises sharply, with both key metrics exceeding the IDE plugin's resource limits.

[0004] Existing technical solutions attempt to address this problem through two paths, but neither has achieved overall optimization. The first approach uses a central repository like SkillsHub, providing only a basic keyword search interface and completely lacking semantic understanding capabilities. As the skill library grows beyond a certain size, the MRR (Mean Responsive Rate) drops sharply. The second approach relies on cloud-based vector databases to build semantic indexes. While offering higher accuracy, it suffers from core drawbacks such as the risk of intellectual property leakage from uploading code data to the cloud, reduced development experience due to network round-trip latency, and inability to support offline use. The consequence is that existing technical solutions fail to achieve a balance between retrieval accuracy, response speed, resource consumption, and privacy protection, thus failing to meet the urgent needs of engineering code in scenarios such as highways for localized, high-precision, and low-latency skill retrieval. Summary of the Invention

[0005] This application provides a high-precision retrieval system for IDE Agent Skills based on a three-layer hybrid architecture, aiming to solve the technical problems of insufficient retrieval accuracy, difficulty in scaling up, excessive resource consumption, and insufficient data privacy protection in the prior art.

[0006] The first aspect of this application provides a high-precision retrieval system for IDE Agent Skills based on a three-layer hybrid architecture, including: The first-level metadata inverted index module is used to build a lightweight inverted index of all Skills metadata. It receives the query string input by the user, calculates the keyword matching relevance score through the field-weighted BM25 scoring model, and reduces the full candidate set to a preset number of candidate Skills within a preset time. The second-layer lightweight semantic vector retrieval module is used to receive the candidate Skills set output by the first layer, perform approximate nearest neighbor retrieval on the pre-computed semantic vectors of the candidate Skills, calculate the cosine similarity between the query vector and the candidate vector, and output a subset of candidate Skills ranked by semantic relevance within a preset time. The third-layer comprehensive scoring fusion module receives the keyword scores from the first layer and the semantic similarity from the second layer, performs a weighted linear fusion calculation on the two scores, generates the final comprehensive ranking result, and outputs the top preset number of search results.

[0007] Furthermore, as one embodiment of the present invention, the first-level metadata inverted index module includes a text preprocessing unit, an index building unit, and a scoring calculation unit; The text preprocessing unit is used to perform Chinese word segmentation, English lowercase conversion, stop word filtering, and English stemming on the raw metadata of each Skill. The index building unit is used to write all Skill documents in batches through the Batch interface of the full-text search engine, and build an inverted index containing the name field, description field, tags field, language field and frameworks field. The name field and description field are segmented using a Chinese-English hybrid word segmenter, while the tags field, language field and frameworks field are segmented using an exact matching word segmenter. The scoring calculation unit is used to calculate the relevance score between query terms and each index field using the field-weighted BM25 scoring formula, where the weight coefficient of the name field is higher than that of the description field, and the weight coefficient of the tags field is set to a medium value.

[0008] Furthermore, as one embodiment of the present invention, the specific implementation of the field-weighted BM25 scoring formula is as follows: Where q is the set of query terms, t is a single query term, and f is the index field. For field weight coefficients, These are word frequency statistics. This is the inverse document frequency statistic. Here, b is the word frequency saturation coefficient, and b is the field length normalization coefficient. For the content of the Skill being retrieved in field f, This represents the average number of words for this field across the entire database.

[0009] Furthermore, as an embodiment of the present invention, the first-level metadata inverted index module also includes a condition combination filtering unit, which is used to implement the combination and nesting of the three logical operators AND, OR, and NOT through the Boolean logic query interface. The filtering conditions include that the language field must match the specified programming language and the tags field must contain the specified tags and not contain deprecated tags.

[0010] Furthermore, as one embodiment of the present invention, the second-layer lightweight semantic vector retrieval module includes an embedding model unit, a vector storage unit, and a similarity calculation unit; The embedding model unit is used to encode the user-input query text in real time through a local lightweight embedding model to generate query semantic vectors. The embedding model adopts a Chinese semantic vector embedding model based on the BERT architecture, which contains multiple Transformer Encoders, with a moderate hidden layer dimension, a small number of parameters, and a model file size in the tens of MB range. The vector storage unit is used to store the semantic vectors of the description and usage_examples fields of each Skill, which are pre-computed during the index building phase. The index structure is constructed using a hierarchical navigable small-world approximate nearest neighbor algorithm, and the 32-bit floating-point vector is compressed into an 8-bit unsigned integer using scalar quantization technology, achieving a compression ratio of 4 to 1. The similarity calculation unit is used to calculate the cosine similarity between the normalized query vector and the normalized candidate vector. Since the pre-calculated candidate vector has been normalized, the cosine similarity calculation is degenerated into a vector dot product operation.

[0011] Furthermore, as an embodiment of the present invention, the parameter configuration of the hierarchical navigable small-world approximate nearest neighbor algorithm includes: setting the maximum number of neighbors of each node in each layer to a moderate value, setting the search width in the construction phase to a large value, setting the search width in the query phase to a medium value, and achieving a retrieval complexity at the logarithmic level.

[0012] Furthermore, as one embodiment of the present invention, the specific implementation steps of scalar quantization compression include: The first step is to traverse the entire vector set and count the minimum and maximum values ​​for each dimension; The second step is to calculate the scaling factor and zero-point offset for each dimension, using the following formula: , ; The third step involves mapping each floating-point value to an 8-bit unsigned integer using a scaling factor and a zero-point offset, and then limiting it to a valid range. The formula is as follows: ; The fourth step involves converting the 8-bit integer value back to a floating-point approximation for distance calculation during the retrieval process. The conversion formula is as follows: .

[0013] Furthermore, as one embodiment of the present invention, the third-layer comprehensive scoring fusion module includes a normalization processing unit, a weighted fusion unit, and an adaptive optimization unit; The normalization unit is used to divide the BM25 score of the first layer output by the maximum value of the BM25 score in the current candidate set to achieve normalization. The weighted fusion unit is used to calculate the final comprehensive ranking score using a weighted linear fusion formula: α and β are weighting coefficients, which are set to equal values ​​by default, and users can adjust the configuration according to the actual usage effect; The adaptive optimization unit is used to periodically collect data on user adoption of search results, aiming to achieve the optimal average reciprocal ranking, and automatically optimizes the value of the weight coefficients through the gradient descent algorithm.

[0014] Furthermore, as one embodiment of the present invention, the gradient descent algorithm of the adaptive optimization unit is specifically implemented as follows: The loss function is defined as the negative reciprocal ranking loss. The ranking gradient is approximated by taking the derivative of the continuous ranking difference. A pair set is constructed for the adopted results and the non-adopted results ranked ahead of the adopted results. After calculating the partial derivative, the weight coefficients are updated according to the gradient descent rule. The constraint that the weight sum is 1 is checked and normalized projection is performed. At the same time, the weight coefficients are restricted to a preset reasonable range.

[0015] Furthermore, as one embodiment of the present invention, the data transfer between the three-layer modules is performed in a synchronous blocking mode. The second layer must wait for the first layer to complete before it can start, and the third layer must wait for the second layer to complete before it can start. The total latency of the three-layer serial execution is controlled within a low range, and the memory peak is controlled within a moderate value.

[0016] Furthermore, as an embodiment of the present invention, the system also includes an incremental index update module, which is used to perform a first-level index segment write operation, a second-level vector graph insertion operation, and a mark deletion strategy when adding a new Skill or modifying an existing Skill, and ensure data consistency through a copy-on-write mechanism.

[0017] The beneficial effects of this invention are: This invention achieves overall optimization in four dimensions—search accuracy, response speed, resource consumption, and privacy protection—through a three-layer progressive hybrid retrieval architecture. The first layer, the metadata inverted index layer, uses a lightweight inverted index to reduce the entire candidate set to the Top 50 range within tens of milliseconds. The second layer, the lightweight semantic vector retrieval layer, performs semantic vector retrieval on the reduced candidate set to avoid the overhead of full computation. The third layer, the comprehensive scoring fusion layer, fully leverages the complementary advantages of keyword matching and semantic understanding through a weighted fusion mechanism, resulting in a significant improvement in the average reciprocal ranking index. At the same time, memory usage is controlled within a moderate range to meet the local resource constraints of IDE plugins. This invention employs a locally deployed lightweight embedded model and full-text search engine. All index and vector data are stored locally on the client, eliminating the need to upload code data to the cloud. This completely eliminates the risk of data leakage and network latency issues, fulfilling the localized processing requirements of code-related data. It can run normally in an offline environment, providing an effective technical solution for the rapid, efficient, and modular development of engineering code, such as smart highways, which require highly customized and localized code. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall technical solution architecture proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the three-layer progressive hybrid retrieval architecture in this invention; Figure 3 This is a logical flow diagram of the first-level metadata inverted index module in this invention; Figure 4 This is a logical flowchart of the second-layer lightweight semantic vector retrieval module in this invention; Figure 5 This is a logical flow diagram of the third-layer comprehensive scoring fusion module in this invention.

[0019] In the attached diagram, the following are the unit labels: 1. First-layer metadata inverted index module; 2. Second-layer lightweight semantic vector retrieval module; 3. Third-layer comprehensive scoring fusion module; 4. Incremental index update module; 11. Text preprocessing unit; 12. Index building unit; 13. Scoring calculation unit; 14. Condition combination filtering unit; 21. Embedding model unit; 22. Vector storage unit; 23. Similarity calculation unit; 31. Normalization processing unit; 32. Weighted fusion unit; 33. Adaptive optimization unit. Detailed Implementation

[0021] To make the objectives, technical solutions, and beneficial effects of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so as to facilitate understanding by those skilled in the art.

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Reference Figure 1 , Figure 2 As shown in the figure, the system includes: a first-layer metadata inverted index module 1, a second-layer lightweight semantic vector retrieval module 2, a third-layer comprehensive scoring fusion module 3, and an incremental index update module 4.

[0024] In this embodiment, the IDE Agent Skills high-precision retrieval system achieves high-precision skill retrieval through a three-layer progressive hybrid retrieval architecture. When a user enters a query string, the system first enters the first-layer metadata inverted index module 1. This module performs keyword matching on all skills using a field-weighted BM25 scoring model, reducing the candidate set from the entire set to a preset number (e.g., Top 50). Subsequently, the candidate skills output from the first layer enter the second-layer lightweight semantic vector retrieval module 2. This module performs approximate nearest neighbor retrieval using pre-computed semantic vectors, calculates the cosine similarity between the query vector and the candidate vectors, and outputs a subset of candidate skills ranked by semantic relevance (e.g., Top 20). Finally, the semantic similarity output from the second layer and the keyword scores output from the first layer enter the third-layer comprehensive score fusion module 3. This module calculates the final comprehensive ranking score through weighted linear fusion and outputs the retrieval results for a preset number (e.g., Top 10). This application achieves overall optimization in four dimensions—search accuracy, response speed, resource consumption, and privacy protection—through this three-layer progressive hybrid retrieval architecture, providing an effective technical solution for scenarios such as smart highways that require highly customized and localized engineering code for rapid, efficient, and modular development.

[0025] In one embodiment, this application also provides a high-precision retrieval system for IDE Agent Skills based on a three-tier hybrid architecture, in which, such as Figure 3 As shown, the first-level metadata inverted index module 1 includes a text preprocessing unit 11, an index building unit 12, a scoring calculation unit 13, and a condition combination filtering unit 14.

[0026] In this embodiment, the first-layer metadata inverted index module 1 is responsible for constructing a lightweight inverted index of all Skills metadata, receiving the query string input by the user, calculating the keyword matching relevance score through a field-weighted BM25 scoring model, and reducing the full candidate set to a preset number of candidate Skills within a preset time. The core objective of this module is to quickly filter a large number of irrelevant candidates through a lightweight inverted index structure while ensuring retrieval recall, significantly reducing the computational overhead of subsequent semantic vector retrieval.

[0027] The text preprocessing unit 11 is used to perform Chinese word segmentation, English lowercase conversion, stop word filtering, and English stemming on the original metadata of each Skill.

[0028] In this embodiment, the processing flow of the text preprocessing unit 11 for Skills metadata includes four key steps: first, the Chinese word segmentation process adopts a dictionary-based forward maximum matching algorithm or a Hidden Markov Model-based probabilistic word segmentation algorithm to split the Chinese description text into meaningful word sequences; second, the English lowercasing process converts all English characters into lowercase forms to eliminate matching deviations caused by case differences; third, the stop word filtering process removes these words from the word segmentation results according to a predefined stop word list (such as high-frequency meaningless words like "de", "le", "the", "is", etc.) to reduce index noise; fourth, the English stemming process adopts the Porter stemming algorithm or the Snowball stemming algorithm to restore English words to their root forms, for example, "running" is restored to "run", and "developers" is restored to "develop". After the above four steps of processing, the original metadata is converted into a standardized term sequence, which provides a unified input for subsequent index construction and scoring calculation.

[0029] The index construction unit 12 is configured to batch write all Skill documents through the Batch interface of the full-text search engine, and construct an inverted index including a name field, a description field, a tags field, a language field and a frameworks field, wherein the name field and the description field are segmented by a Chinese-English mixed word segmenter, and the tags field, the language field and the frameworks field are segmented by an exact matching word segmenter.

[0030] In this embodiment, the index building unit 12 adds preprocessed Skill documents to the index in batches through the batch write interface of a full-text search engine (such as Apache Lucene, Elasticsearch, or OpenSearch). Each Skill document contains five core index fields: the name field stores the skill name, the description field stores the skill's functional description, the tags field stores the skill tags, the language field stores the supported programming languages, and the frameworks field stores the supported development frameworks. The name and description fields are segmented using a mixed Chinese-English word segmenter. This segmenter has a built-in Chinese and English dictionary, can correctly identify word boundaries in mixed Chinese and English text, and supports mixed segmentation including Chinese characters, English words, numbers, and special characters. The tags, language, and frameworks fields are segmented using an exact matching word segmenter. The content of these fields is usually discrete tag values ​​(such as "python", "web", "REST"), which do not need to be segmented; maintaining the original form is beneficial for exact matching queries.

[0031] The specific process of index building is as follows: First, an index schema is created, defining the name, type, and tokenizer configuration of each field. Then, the entire Skills metadata is traversed, and the document objects of each Skill are written to the index segments in batches using the `addDocuments` method of `IndexWriter`. During the writing process, the full-text search engine maintains an inverted index data structure internally, recording the document list corresponding to each term and its frequency, position, and other information. After the index is built, the system automatically performs a segment merge operation, merging multiple small segments into fewer segments to optimize the efficiency of subsequent queries.

[0032] The scoring calculation unit 13 is used to calculate the relevance score between the query term and each index field using the field-weighted BM25 scoring formula, wherein the weight coefficient of the name field is higher than that of the description field, and the weight coefficient of the tags field is set to a medium value.

[0033] In this embodiment, the scoring calculation unit 13 uses a field-weighted BM25 scoring model to calculate the relevance score between the query terms and each index field. BM25 (Best Matching 25) is a classic term matching retrieval algorithm that evaluates the relevance between a document and a query by statistically analyzing term frequency and inverse document frequency. The specific implementation of the field-weighted BM25 scoring formula is as follows: in, To query the set of terms, For a single query term, For index fields, For field weight coefficients, These are word frequency statistics. This is the inverse document frequency statistic. This is the word frequency saturation coefficient (usually ranging from 1.2 to 2.0). This is the field length normalization coefficient (usually 0.75). For the Skill being retrieved in the field The length of the content in the middle, This represents the average number of words for this field across the entire database.

[0034] The field weighting strategy is as follows: the name field is set to the highest weighting value (e.g., 2.0), because the skill name usually reflects the functional characteristics of the skill most directly; the description field is set to a medium weighting value (e.g., 1.0), as it serves as the main carrier of functional description; the tags field is set to a medium-low weighting value (e.g., 0.8), because tag information is concise but has limited coverage; the language and frameworks fields are set to low weighting values ​​(e.g., 0.5), mainly used as filtering conditions rather than sorting criteria.

[0035] Inverse document frequency The calculation formula is: in, This represents the total number of documents in the index. For included terms The number of documents. The higher the IDF value, the stronger the distinguishing ability of the term, and the more effectively it can filter relevant documents.

[0036] The execution flow of the scoring calculation unit 13 is as follows: First, it receives the query string input by the user and obtains the query term sequence through the same processing flow of the text preprocessing unit 11; Second, for each candidate Skill document, it iterates through each term in the query term sequence and searches for the occurrence of the term in each index field; Third, it calculates the score of each field according to the field weighted BM25 formula; Fourth, it sums the scores of each field by weight to obtain the final BM25 score; Fifth, it sorts the scores in descending order and outputs a preset number (e.g., Top 50) of candidate Skills.

[0037] The condition combination filtering unit 14 is used to implement the combination and nesting of the three logical operators AND, OR and NOT through the Boolean logic query interface. The filtering conditions include that the language field must match the specified programming language and the tags field must contain the specified tags and not contain deprecated tags.

[0038] In this embodiment, the condition combination filtering unit 14 provides flexible Boolean logic query capabilities, allowing developers to precisely filter Skills by combining conditions. This unit supports three basic logical operators: the AND operator indicates that multiple conditions are met simultaneously (e.g., language:python AND tags:web), the OR operator indicates that any one condition is met (e.g., tags:database OR tags:cache), and the NOT operator excludes documents that meet specific conditions (e.g., NOT tags:deprecated). These three operators can be nested and combined to construct complex query expressions.

[0039] The specific implementation of the filtering conditions includes: First, the `language` field must match the specified programming language. For example, when querying Skills related to "python", only documents whose `language` field contains "python" will be retained. Second, the `tags` field must contain the specified tags. For example, Skills with the tag "REST API" are required. Third, the `tags` field must not contain deprecated tags. For example, documents with tags containing "deprecated" or "obsolete" will be excluded. The condition combination filtering unit 14 implements the above functions through the BooleanQuery interface of the full-text search engine, supporting multi-level nested query condition combinations.

[0040] In one implementation, this application also provides a high-precision retrieval system for IDE Agent Skills based on a three-layer hybrid architecture, such as... Figure 4 As shown, in this system, the second-layer lightweight semantic vector retrieval module 2 includes an embedding model unit 21, a vector storage unit 22, and a similarity calculation unit 23.

[0041] In this embodiment, the second-layer lightweight semantic vector retrieval module 2 receives the candidate Skills set output from the first layer, performs approximate nearest neighbor retrieval on the pre-computed semantic vectors of the candidate Skills, calculates the cosine similarity between the query vector and the candidate vector, and outputs a subset of candidate Skills ranked by semantic relevance within a preset time. The core objective of this module is to perform deep semantic understanding on the reduced candidate set from the first layer, compensating for the inability of keyword matching to capture semantic similarity.

[0042] The embedding model unit 21 is used to encode the query text input by the user in real time through a local lightweight embedding model to generate a query semantic vector. The embedding model adopts a Chinese semantic vector embedding model based on the BERT architecture, which contains multiple layers of Transformer Encoders, with a moderate hidden layer dimension, a small number of parameters, and a model file size in the tens of MB range.

[0043] In this embodiment, the embedding model unit 21 adopts a Chinese semantic vector embedding model based on the BERT (Bidirectional Encoder Representations from Transformers) architecture. This model can encode Chinese or mixed Chinese-English text of arbitrary length into dense vectors of fixed dimensions. The specific architecture of the model includes the following components: the input layer is responsible for converting the original text into a combination of word embedding, position embedding, and token type embedding; the multi-layer TransformerEncoder is responsible for capturing the contextual semantic information in the text through a self-attention mechanism. Each layer contains two sub-layers: multi-head attention and a feed-forward network. Residual connections and layer normalization are used to stabilize the training process; the output layer is responsible for aggregating the output of the last layer of the Transformer Encoder. Common aggregation strategies include CLS token embedding (directly using the vector corresponding to the CLS token as the sentence representation) or Mean Pooling (averaging the vectors of all tokens).

[0044] The model parameter control strategy is as follows: the hidden layer dimension is set to a moderate value (e.g., 256 or 384), which is significantly reduced compared to the 768 dimensions of the standard BERT-base; the number of Transformer Encoder layers is set to 4 to 6, which is reduced compared to the 12 layers of the standard BERT-base; the number of attention heads is set to 4 to 8; and the feedforward network dimension is set to 2 to 4 times the hidden layer dimension. With these parameter configurations, the model file size is controlled to the tens of MB level (e.g., 40MB to 80MB), meeting the local resource constraints of the IDE plugin and allowing it to run normally in an offline environment.

[0045] The inference process of the embedding model unit 21 is as follows: First, the query text input by the user is preprocessed, including word segmentation and adding special tokens (CLS, SEP), etc.; Second, the preprocessed input sequence is converted into the tensor format required by the model, including token IDs, attention mask and token type IDs; Third, the tensor is input into the embedding model, passing through the embedding layer and the TransformerEncoder layer in sequence; Fourth, the sentence representation vector (CLS token or Mean Pooling result) is extracted from the model output; Fifth, the sentence representation vector is L2 normalized to generate the final query semantic vector.

[0046] The vector storage unit 22 is used to store the semantic vectors of the description field and usage_examples field of each Skill pre-computed during the index building stage. The index structure is constructed using a hierarchical navigable small-world approximate nearest neighbor algorithm, and the 32-bit floating-point vector is compressed into an 8-bit unsigned integer using scalar quantization technology, achieving a compression ratio of 4 to 1.

[0047] In this embodiment, vector storage unit 22 is responsible for storing and managing the pre-computed semantic vectors of all Skills. During the index building phase, the system pre-encodes the description field (functional description) and usage_examples field (usage examples) of each Skill using the aforementioned embedding model, generating and storing the corresponding semantic vectors. Vector storage unit 22 uses the Hierarchical Navigable Small World (HNSW) algorithm to build the index structure, which can achieve efficient approximate nearest neighbor search in high-dimensional vector space.

[0048] The core principle of the HNSW algorithm is as follows: The algorithm constructs a multi-layered index structure, where each layer is an approximate nearest neighbor graph based on a Navigable Small World (NSW) graph structure. The bottom layer contains the most complete neighbor connections, has the largest number of edges, and achieves the highest search accuracy; the upper layers have fewer edges, resulting in faster search speeds. During a query, starting from the top layer, a greedy search is performed along the edges, gradually approaching the location of the target vector, and finally, an accurate search is completed at the bottom layer.

[0049] The parameter configuration of the HNSW algorithm includes: First, the maximum number of neighbors (efConstruction or M) of each node in each layer is set to a moderate value (e.g., 16 or 32) to control the connection density of the graph; Second, the search width (efConstruction) in the construction phase is set to a relatively large value (e.g., 100 to 200) to ensure index quality; Third, the search width (ef) in the query phase is set to a medium value (e.g., 10 to 50) to achieve a balance between search efficiency and accuracy; Fourth, the retrieval complexity reaches the logarithmic level, that is, the query time complexity is O(log N), where N is the total number of vectors.

[0050] Vector storage unit 22 also employs scalar quantization (SQ) technology to compress and store vectors. Specific implementation steps include: The first step is to traverse the entire vector set and count the minimum (min) and maximum (max) of each dimension. The second step is to calculate the scaling factor and zero offset for each dimension, using the following formula: , ; The third step is to map each floating-point value to an 8-bit unsigned integer (range 0 to 255) using a scaling factor and a zero-point offset, as shown in the formula. ; The fourth step is to convert the 8-bit integer value back to a floating-point approximation for distance calculation during the retrieval process. The conversion formula is as follows: .

[0051] Through scalar quantization, a 32-bit floating-point vector (4 bytes) is compressed into an 8-bit unsigned integer (1 byte), achieving a compression ratio of 4 to 1, which significantly reduces storage overhead and memory usage.

[0052] The similarity calculation unit 23 is used to calculate the cosine similarity between the normalized query vector and the normalized candidate vector. Since the pre-calculated candidate vector has been normalized, the cosine similarity calculation is degenerated into a vector inner product operation.

[0053] In this embodiment, the similarity calculation unit 23 is responsible for calculating the semantic similarity between the query vector and the candidate vector. Cosine similarity is used as the similarity metric, and its calculation formula is as follows: in, and They are two vectors, For vector dimensions, This represents the L2 norm.

[0054] Since the candidate vectors pre-computed in vector storage unit 22 have already undergone L2 normalization (i.e. Furthermore, the query vector generated by the embedded model unit 21 has also undergone L2 normalization (i.e., Cosine similarity calculation can be simplified to a vector dot product operation: This simplification significantly reduces computational complexity by eliminating square root and division operations, resulting in a clear performance advantage in large-scale vector retrieval scenarios.

[0055] The execution flow of the similarity calculation unit 23 is as follows: First, it receives the normalized query vector generated by the embedding model unit 21; second, it performs an approximate nearest neighbor search through the HNSW index structure to obtain a preset number of candidate vectors (e.g., Top 100) and their index IDs; third, it calculates the inner product of the query vector and each candidate vector to obtain the cosine similarity score; fourth, it sorts the candidate skills in descending order of similarity score and outputs a preset number of candidate skills (e.g., Top 20) and their similarity scores.

[0056] In one implementation, this application also provides a high-precision retrieval system for IDE Agent Skills based on a three-layer hybrid architecture, such as... Figure 5 As shown, in this system, the third-layer comprehensive scoring fusion module 3 includes a normalization processing unit 31, a weighted fusion unit 32, and an adaptive optimization unit 33.

[0057] In this embodiment, the third-layer comprehensive scoring fusion module 3 receives the keyword scores output from the first layer and the semantic similarity output from the second layer, performs a weighted linear fusion calculation on the two scores, generates the final comprehensive ranking result, and outputs the top preset number of search results. This module fully leverages the complementary advantages of keyword matching and semantic understanding as two retrieval methods to achieve a more accurate ranking effect.

[0058] The normalization processing unit 31 is used to divide the BM25 score output by the first layer by the maximum value of the BM25 score in the current candidate set to achieve normalization processing.

[0059] In this embodiment, since the numerical distribution ranges of the first-layer BM25 score and the second-layer cosine similarity are different, directly performing weighted fusion will lead to one dimension dominating the ranking result. The normalization processing unit 31 is responsible for unifying the two scores to the same numerical range, using the maximum value normalization method to map the BM25 score to the [0,1] interval: in, The BM25 score for the current candidate Skill. This is the maximum BM25 score for all Skills in the current candidate set. The normalized BM25 score eliminates dimensional differences, providing a unified basis for subsequent weighted fusion.

[0060] The execution flow of the normalization processing unit 31 is as follows: First, receive the BM25 scores of all candidate Skills output from the first layer; second, traverse the score set, identify and record the maximum value. The third step is to divide the BM25 score of each candidate Skill by... The normalized score is obtained; the fourth step is to pass the normalized result to the weighted fusion unit 32.

[0061] The weighted fusion unit 32 is used to calculate the final comprehensive ranking score using a weighted linear fusion formula: α and β are weighting coefficients, which are set to equal values ​​by default, and users can adjust the configuration according to the actual usage effect.

[0062] In this embodiment, the weighted fusion unit 32 adopts a weighted linear fusion strategy, which sums the normalized BM25 score and the cosine similarity in a weighted manner to obtain the final comprehensive ranking score: Where α is the keyword matching weight, β is the semantic similarity weight, and their sum is usually set to 1. Under the default configuration, This indicates that the two search methods are equally important. In practical applications, users can adjust the weighting ratio according to the specific scenario: if priority is given to exact keyword matching, the α value can be increased (e.g., 0.7); if priority is given to semantic relevance, the β value can be increased (e.g., 0.7).

[0063] The execution flow of the weighted fusion unit 32 is as follows: First, it receives the normalized BM25 score output by the normalization processing unit 31 and the cosine similarity output by the similarity calculation unit 23; second, it calculates the weighted fusion score according to the preset or adaptively optimized weight coefficients α and β; third, it follows the... Sort the results in descending order and output the search results for a preset number of Top results (e.g., Top 10).

[0064] The adaptive optimization unit 33 is used to periodically collect data on user adoption of search results, aiming to achieve the optimal Mean Reciprocal Rank (MRR), and automatically optimizes the value of the weight coefficients through the gradient descent algorithm.

[0065] In this embodiment, the adaptive optimization unit 33 automatically optimizes the fusion weights using machine learning methods to improve the ranking quality of the retrieval system. This unit periodically collects data on user adoption of search results; for example, if a user clicks on the third search result, it indicates that the third result is a relevant document. The Mean Reverse Rank (MRR) is a commonly used metric for measuring ranking quality, and its calculation formula is as follows: in, To query the total number, For the first The ranking position of the first relevant document in a query. A higher MRR value indicates better ranking quality.

[0066] The gradient descent algorithm of adaptive optimization unit 33 is implemented as follows: The first step is to define the loss function as the Negative Reciprocal Rank Loss, which measures the deviation of the ranking quality from the optimal ranking: in, The position of the search results selected for the user in the sorted list.

[0067] The second step approximates the ranking gradient by taking the derivative of the continuous ranking differences. For each adopted result and a pairwise set of non-adopted results ranked ahead of them, the partial derivatives are calculated, and the weight coefficients are updated according to the gradient descent rule. Let the... The fusion scores of the candidates are ,in To normalize the BM25 score, This represents the cosine similarity. (Regarding the adopted results...) Non-adoption results ( Construct the loss function: in, The marginal margin is (usually set to 0.1). The partial derivative of the loss function with respect to the weight coefficients is: in, This is an indicator function that takes the value 1 when the condition is true and 0 otherwise.

[0068] The third step is to update the weight coefficients according to the gradient descent rule: in, This is the learning rate (typically set to 0.01 to 0.1).

[0069] Fourth step: Check the constraints with a weight sum of 1 and perform normalized projection: The fifth step is to limit the weight coefficients to a preset reasonable range (e.g., [0.1, 0.9]) to prevent a certain weight from approaching 0 or 1 and causing a single search method to dominate the results.

[0070] The execution flow of the adaptive optimization unit 33 is as follows: First, periodically collect user adoption data, including query statements, candidate lists, and documents clicked by users; Second, construct training samples, and for each adoption result, select the non-adoption results ranked ahead of it as negative samples; Third, calculate the gradient and update the weight coefficients; Fourth, periodically synchronize the updated weight coefficients to the weighted fusion unit 32.

[0071] In one embodiment, this application also provides a high-precision retrieval system for IDE Agent Skills based on a three-layer hybrid architecture. In this system, data transfer between the three-layer modules is performed in a synchronous blocking mode. The second layer must wait for the first layer to complete before it can start, and the third layer must wait for the second layer to complete before it can start. The total latency of the three-layer serial execution is controlled within a low range, and the peak memory usage is controlled within a moderate value.

[0072] In this embodiment, to ensure the correctness of the search results and the stability of the system, the three-layer modules are executed in a synchronous blocking mode. The specific execution process is as follows: First, after the user inputs the query string, the system first calls the first-layer metadata inverted index module 1 to perform keyword matching and retrieval. After the module is completed, it outputs the Top 50 candidate skills and their BM25 scores. Second, the system takes the candidate skills output by the first layer as input and calls the second-layer lightweight semantic vector retrieval module 2 to perform semantic vector retrieval. After the module is completed, it outputs the Top 20 candidate skills and their cosine similarity. Third, the system takes the BM25 score of the first layer and the cosine similarity of the second layer as input and calls the third-layer comprehensive score fusion module 3 to perform weighted fusion. After the module is completed, it outputs the Top 10 final search results.

[0073] The advantages of synchronous blocking mode are: the input data of each layer is only used after the previous layer has been fully executed, avoiding data race and state inconsistency problems; the execution flow is clear, which is convenient for debugging and performance analysis; a timeout mechanism can be set in each layer, and when the execution time of a certain layer exceeds the threshold, it will terminate in time and return part of the result.

[0074] In terms of performance control, total latency is kept within a low range (typically 50ms to 200ms) to meet the real-time requirements of the IDE plugin; peak memory usage is controlled at a moderate level (typically 200MB to 500MB) to meet the local resource constraints of the IDE plugin. The three-layer serial execution pipeline design ensures efficient utilization of system resources and avoids the additional overhead and complexity brought by parallel execution.

[0075] In one embodiment, this application also provides a high-precision retrieval system for IDE Agent Skills based on a three-layer hybrid architecture. In this system, the system further includes an incremental index update module 4, which is used to perform a first-layer index segment write operation, a second-layer vector graph insertion operation, and a mark-deletion strategy when adding a new Skill or modifying an existing Skill, and ensures data consistency through a copy-on-write mechanism.

[0076] In this embodiment, the incremental index update module 4 is responsible for dynamically updating the index data at runtime, supporting the addition, modification, and deletion of Skills without rebuilding the entire index. This module ensures data consistency through a copy-on-write (COW) mechanism, ensuring that ongoing retrieval requests are not affected during the index update process.

[0077] The execution flow of incremental index update module 4 is as follows: The first stage involves index segment writing. When a new Skill is added, the system first performs text preprocessing on the metadata of the new Skill (same as in the index building stage) to generate a normalized term sequence. Then, the new document is written to a new index segment through the incremental write interface (IndexWriter.addDocument) of the full-text search engine. At the same time, the description and usage_examples fields of the new Skill are encoded using an embedding model to generate semantic vectors, and the vectors are added to the HNSW graph structure through the vector index insertion interface.

[0078] The second stage involves a mark-and-delete strategy. When modifying or deleting an existing Skill, the system does not directly modify the existing index segment (because index segments are immutable). Instead, it uses a mark-and-delete strategy: First, a deletion operation is performed in the full-text search engine (IndexWriter.deleteDocument), marking the document to be deleted as deleted based on the document ID or query conditions. Second, a deletion operation is performed in the vector index, marking the corresponding vector as deleted. Third, in subsequent queries, the search engine automatically filters out documents marked as deleted.

[0079] The third stage is the copy-on-write mechanism. To ensure data consistency, the system adopts a copy-on-write strategy: First, when an index update operation is performed, the system creates a snapshot of the index; second, retrieval requests are executed based on the snapshot, ensuring that ongoing queries are not affected during the update process; third, after the update operation is completed, the resources of the old snapshot are released; fourth, the newly written data becomes visible to subsequent retrieval requests.

[0080] The periodic compaction mechanism of incremental index update module 4 is as follows: As deletion operations increase, a large number of deleted documents accumulate in the index, affecting query efficiency and storage space. The system periodically performs a segment merging operation (IndexWriter.forceMerge) to merge multiple small segments into fewer segments, while simultaneously cleaning up deleted documents and freeing up storage space. The merging process is performed in the background and does not block foreground retrieval requests.

[0081] Thus, in the IDE Agent Skills high-precision retrieval system based on a three-layer hybrid architecture, this application achieves efficient keyword matching by using a lightweight inverted index module 1 (first-layer metadata inverted index module 1) to reduce the entire candidate set to the Top 50 range within tens of milliseconds. The second-layer lightweight semantic vector retrieval module performs semantic vector retrieval on the reduced candidate set, avoiding full computational overhead and achieving deep semantic understanding. The third-layer comprehensive scoring fusion module's weighted fusion mechanism fully leverages the complementary advantages of keyword matching and semantic understanding, significantly improving the average reciprocal ranking index. Simultaneously, memory usage is controlled within a moderate range, meeting the local resource constraints of the IDE plugin. This invention employs a locally deployed lightweight embedding model and full-text search engine, storing all index and vector data locally on the client, eliminating the need to upload code data to the cloud, completely eliminating data leakage risks and network latency issues, and fulfilling the localized processing requirements of code-related data. It can operate normally in an offline environment, providing an effective technical solution for the rapid, efficient, and modular development of highly customized and localized engineering code, such as smart highways.

[0082] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0083] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. An IDE Agent Skills high-precision retrieval system based on a three-layer hybrid architecture, characterized by: include: The first-level metadata inverted index module (1) is used to receive the query string input by the user, calculate the keyword matching relevance score through the field weighted BM25 scoring model, and reduce the full candidate set to the candidate Skills within a preset time. The second-layer lightweight semantic vector retrieval module (2) is used to receive the candidate Skills set output by the first-layer metadata inverted index module (1), perform approximate nearest neighbor retrieval on the pre-computed semantic vectors of the candidate Skills, calculate the cosine similarity between the query vector and the candidate vector, and output the candidate Skills subset sorted by semantic relevance within a preset time. The third-layer comprehensive scoring fusion module (3) is used to receive the keyword scores output by the first-layer metadata inverted index module (1) and the semantic similarity output by the second-layer lightweight semantic vector retrieval module (2), perform weighted linear fusion calculation on the two scores, generate the final comprehensive ranking result and output the top preset number of retrieval results; wherein, the second-layer lightweight semantic vector retrieval module (2) is started in response to the first-layer metadata inverted index module (1) after processing, and the third-layer comprehensive scoring fusion module (3) is started in response to the second-layer lightweight semantic vector retrieval module (2) after processing.

2. The system according to claim 1, characterized in that, The first layer metadata inverted index module (1) includes a text preprocessing unit (11), an index building unit (12), and a scoring calculation unit (13). The text preprocessing unit (11) is used to perform Chinese word segmentation, English lowercase conversion, stop word filtering, and English stemming on the original metadata of each Skill. The index building unit (12) is used to write all Skill documents in batches through the Batch interface of the full-text search engine to build an inverted index containing the name field, description field, tags field, language field, and frameworks field. The scoring calculation unit (13) is used to calculate the relevance score between the query terms and each index field using the field-weighted BM25 scoring formula.

3. The system according to claim 2, characterized in that, In the index building unit 12, the name and description fields are segmented using a mixed Chinese and English word segmenter, while the tags, language, and frameworks fields are segmented using an exact matching word segmenter.

4. The system according to claim 2, characterized in that, In the scoring calculation unit (13), the weight coefficient of the name field is higher than that of the description field, and the weight coefficient of the tags field is set to a medium value.

5. The system according to claim 1, characterized in that, The first layer metadata inverted index module (1) also includes a condition combination filtering unit (14), which is used to implement the combination and nesting of three logical operators AND, OR and NOT through the Boolean logic query interface. The filtering conditions include that the language field must match the specified programming language and the tags field must contain the specified tags and not contain deprecated tags.

6. The system according to claim 1, characterized in that, The second-layer lightweight semantic vector retrieval module (2) includes an embedding model unit (21), a vector storage unit (22), and a similarity calculation unit (23). The embedding model unit (21) is used to encode the query text input by the user in real time through a local lightweight embedding model to generate a query semantic vector. The vector storage unit (22) is used to store the pre-calculated semantic vectors of the description field and usage_examples field of each Skill. The index structure is constructed using a hierarchical navigable small-world approximate nearest neighbor algorithm, and the 32-bit floating-point vector is compressed into an 8-bit unsigned integer using scalar quantization technology. The similarity calculation unit (23) is used to calculate the cosine similarity between the normalized query vector and the normalized candidate vector. The cosine similarity calculation is degenerated into a vector inner product operation.

7. The system according to claim 6, characterized in that, In the vector storage unit (22), the parameter configuration of the hierarchical navigable small-world approximate nearest neighbor algorithm includes: the maximum number of neighbors of each node in each layer is set to a moderate value, the search width in the construction phase is set to a large value, and the search width in the query phase is set to a medium value.

8. The system according to claim 6, characterized in that, In the vector storage unit (22), scalar quantization compression includes the following steps: traversing the full vector set to count the minimum and maximum values ​​of each dimension; calculating the scaling factor and zero offset for each dimension; mapping each floating-point value to an 8-bit unsigned integer using the scaling factor and zero offset and limiting it to a valid range; and restoring the 8-bit integer value to a floating-point approximation for distance calculation during retrieval.

9. The system according to claim 1, characterized in that, The third-layer comprehensive scoring fusion module (3) includes a normalization processing unit (31), a weighted fusion unit (32), and an adaptive optimization unit (33). The normalization processing unit (31) is used to divide the BM25 score output by the first-layer metadata inverted index module (1) by the maximum value of the BM25 score in the current candidate set to achieve normalization processing. The weighted fusion unit (32) is used to calculate the final comprehensive ranking score through the weighted linear fusion formula: Scorefinal=α×normalized BM25 score+β×cosine similarity, where α and β are weight coefficients. The adaptive optimization unit (33) is used to periodically collect data on the adoption of search results by users, aiming to achieve the optimal average reciprocal ranking, and automatically optimize the value of the weight coefficients through the gradient descent algorithm.

10. The system according to claim 1, characterized in that, It also includes an incremental index update module (4), which is used to perform first-level index segment write operations, second-level vector graph insertion operations and mark deletion strategies when adding a new Skill or modifying an existing Skill, and ensure data consistency through a copy-on-write mechanism.

Images