A semantic core weighted query method, device and medium for retrieval enhancement generation

Abstract

The application discloses a kind of retrieval enhancement generation-oriented semantic core weighting query method, device and medium, comprising: input query text is matched with the block in external knowledge base and is filtered and sorted, obtains front candidate block;Cosine similarity of candidate word in query text and the word in candidate block is obtained and thus filtering is completed, forms semantic neighbor, candidate word and semantic neighbor constitute semantic cluster;Design twice mask process to calculate semantic similarity drop, obtain semantic sensitivity score, select core word list and its sensitivity score, select core word according to score order;Optimize the branch of sparse retrieval and dense retrieval;The block searched by the two branches after optimization is reordered, the most similar block in external knowledge base, collate and output final result.The application can be without manual annotation and additional LLM power, through semantic core weighting targeted solution retrieval optimization scheme, replace artificial to some extent, improve efficiency.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, specifically to a semantic core weighted query method, apparatus, and medium for enhanced retrieval generation. Background Technology

[0002] Large language models, with their powerful contextual modeling capabilities, have achieved accurate semantic understanding, effective knowledge modeling, and high-quality content generation in generative tasks. Their applications have permeated natural language processing, vertical industry services, and multimodal interaction, attracting widespread attention. However, limited by static training data, large models are prone to knowledge illusions or information lag when dealing with scenarios requiring high knowledge timeliness or dense knowledge, severely impacting the quality of generated content. Retrieval-enhanced generation (RAG) significantly improves the accuracy of generated content by dynamically retrieving relevant knowledge fragments from external authoritative knowledge bases and accurately integrating them into the generated input pattern.

[0003] The retrieval enhancement technology follows a three-step closed-loop process of retrieval, fusion, and output. The retrieval module locates knowledge fragments highly relevant to the query from heterogeneous external knowledge bases; the fusion module integrates external knowledge with the core requirements of the query to achieve organic integration of the large model's semantic framework; and the output module generates content based on the fused knowledge. Among these, the retrieval module directly determines the quality of subsequent steps and is the core bottleneck of the RAG system.

[0004] Traditional retrieval enhancement methods, such as TF-IDF and BM25, rely on word frequency statistics for representation and weighting, but lack the ability to filter out distracting words. This not only easily falls into the misconception that high frequency equals importance, but also obscures key query chunks with high-frequency, non-critical information. Dense retrieval methods, such as Sentence-BERT, improve semantic capture capabilities, but their mechanism of uniformly representing entire sentence vectors struggles to distinguish word importance and requires fine-tuning with labeled data, resulting in weak generalization in zero-training scenarios. Hybrid retrieval methods, such as Faiss+BM25, combine sparse and dense features, but lack a mechanism for explicitly linking statistical indicators with semantic importance. In long query scenarios, the negative impact of distracting words significantly reduces retrieval accuracy. Existing query retrieval methods suffer from drawbacks such as reliance on manual annotation and high computational demands. Furthermore, existing approaches fail to effectively address the issues of accidental deletion of low-frequency core words and the disconnect between semantics and statistical indicators. Unsupervised, high-efficiency retrieval optimization solutions remain lacking. Summary of the Invention

[0005] The purpose of this invention is to provide a semantic core weighted query method, apparatus, and medium for enhanced retrieval generation, in order to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A semantic core-weighted query method for retrieval enhancement generation includes the following steps:

[0008] Step 1: Obtain the input Chinese query text and match, filter and sort it with the language blocks in the external knowledge base to obtain the top-ranked candidate language blocks;

[0009] Step 2: Obtain the cosine similarity between candidate words and words in candidate blocks in the query text and filter accordingly to form semantic neighbors. Candidate words and the obtained semantic neighbors together constitute semantic clusters, forming a candidate word-semantic cluster mapping relationship.

[0010] Step 3: Design two masking processes to calculate the semantic similarity reduction, obtain the semantic sensitivity score, filter out the core word list and the corresponding semantic sensitivity score, and select the core words according to the semantic sensitivity score.

[0011] Step 4: Optimize the sparse retrieval branch by using an improved sparse algorithm that integrates semantic sensitivity scores and inverse document frequency; optimize the dense retrieval branch by generating core word scores through a quantization function, and then weighting the original query vector based on the core word scores to obtain an enhanced query vector for vector retrieval.

[0012] Step 5: The language chunks retrieved by the optimized sparse search branch and the optimized dense search branch are reordered through the result fusion output module. The language chunks that are most similar to the query language chunks are obtained from the external knowledge base, and the final results are sorted and output.

[0013] Furthermore, in step 1, chunk matching and filtering are performed using semantic vector matching and keyword search methods, and the chunks are sorted according to their relevance.

[0014] Furthermore, in step 2, after calculating the cosine similarity, the top few similar words matched by each candidate word are selected based on the obtained cosine similarity, and then words below the threshold ω are filtered out, and the retained words are taken as semantic neighbors.

[0015] Further, step 3 includes:

[0016] Step 3.1: Mask only the candidate words, calculate the semantic similarity difference between the query text and the target text before and after masking, and denote it as △Sim1;

[0017] Step 3.2: Mask the candidate words and their semantic clusters simultaneously, and calculate the semantic similarity difference, denoted as △Sim2;

[0018] Step 3.3, define the semantic sensitivity score as S_sem, and its formula is:

[0019]

[0020] Where α is the weight value of the semantic contribution of the candidate word itself and the joint contribution of the semantic cluster;

[0021] Step 3.4: Select the top N words with the highest semantic sensitivity scores for each query term as core words, that is, select the most important words in the entire query block, where N is a natural number greater than 0.

[0022] Furthermore, in step 4, the formula for the relevance score between the document and the core keywords is:

[0023]

[0024] Wherein, TF(bool) represents the word frequency count value, with 0 / 1 marking whether the word appears; S_sem is the semantic sensitivity score; and IDF(w) reflects the statistical scarcity of the target word w.

[0025] Furthermore, in step 4, the optimization of dense retrieval is based on a query vector enhancement strategy weighted by core word scores. By integrating semantic importance and statistical scarcity, it achieves accurate retrieval optimization within the vector space. The core of dense retrieval optimization includes two parts: a quantization function and a vector enhancement mechanism. The quantization function is defined as CoreScore(w), and its formula is:

[0026]

[0027] Where △sim(w) represents the decrease in semantic similarity between the query statement and the candidate block after the target word w is masked; 1-△sim(w) represents the semantic irreplaceability of the target word w; log(IDF(w)+1) represents the statistical weight of low-frequency words strengthened by logarithmic transformation;

[0028] The vector augmentation mechanism is based on pre-trained word embeddings. It involves weighted superposition of core word vectors and dragging the query vector in the pre-trained vector space towards a phrase containing the core words, resulting in a weighted query vector V_q' of the original query vector V_q. The formula is as follows:

[0029]

[0030] Where V_w is the word vector extracted from the word embedding layer of the pre-trained model, which contains the contextual semantic information of the word, and λ is an adjustable parameter for the weighted vector strength of the core word.

[0031] The present invention also provides a semantic core weighted query apparatus for retrieval enhancement generation, comprising one or more processors for implementing a semantic core weighted query method for retrieval enhancement generation as described above.

[0032] The present invention also provides a readable storage medium storing a program that, when executed by a processor, implements a semantic core weighted query method for retrieval enhancement generation as described above.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] 1) The semantic core weighted query enhancement method proposed in this invention can solve the retrieval optimization scheme in a targeted manner through semantic core weighting without manual annotation and additional LLM computing power, thus replacing manual work to a certain extent and improving efficiency.

[0035] 2) The semantic core weighted query enhancement method proposed in this invention effectively solves the problems of accidental deletion of low-frequency core words and the disconnect between semantics and statistical indicators, and reduces the scenario where the search accuracy is significantly reduced due to the negative impact of interference words.

[0036] 3) The semantic core weighted query enhancement method proposed in this invention quantifies the semantic irreplaceability of words through the "mask-probe" mechanism, and participates the quantified values ​​in CoreScore weighted query vector enhancement and keyword retrieval, thereby improving the effectiveness of the retrieval system. Attached Figure Description

[0037] Figure 1 This is a flowchart of a semantic core weighted query method for enhanced retrieval generation according to the present invention.

[0038] Figure 2 This is a flowchart of the optimized sparse retrieval process in the semantic core weighted query method for retrieval enhancement generation of the present invention.

[0039] Figure 3 This is a flowchart of the optimized dense retrieval process in the semantic core weighted query method for retrieval enhancement generation of the present invention.

[0040] Figure 4 This is a schematic diagram of the semantic core weighted query device for enhanced retrieval generation according to the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Reference Figure 1 A semantic core weighted query method for enhanced retrieval generation, the method comprising the following steps:

[0043] Step 1: Obtain the input Chinese query text through semantic vector matching and match it with the language blocks in the external knowledge base to filter and sort them, and obtain the top 10 candidate language blocks.

[0044] Step 2: Calculate the cosine similarity between each candidate word in the query text and all words in the candidate block. Based on the obtained cosine similarity, select the top 3 most similar words that match each candidate word, then filter out words with a similarity below the threshold of 0.5, and keep the remaining words as semantic neighbors.

[0045] Step 3: Design two masking processes to calculate the semantic similarity reduction, obtain the semantic sensitivity score S_sem, filter out the core word list and corresponding S_sem values, and select core words according to the S_sem values, including:

[0046] Step 3.1: Mask only the candidate words (i.e. replace them with special tags), calculate the semantic similarity difference between the query text and the target text before and after masking, and denot it as △Sim1.

[0047] Step 3.2: Mask the candidate words and their semantic clusters (same semantic words) simultaneously, and calculate the semantic similarity difference, denoted as △Sim2.

[0048] Step 3.3 above describes the two semantic similarity differences obtained through the "mask-probe" mechanism. The similarity difference can be used to represent the impact of masking a word on the entire text. Then, a semantic sensitivity score is obtained by balancing the semantic contribution of the candidate word itself with the joint contribution of the semantic cluster, thus quantifying the semantic non-substitutability of the word. The semantic sensitivity score is defined as S_sem, and its formula is:

[0049] (1)

[0050] Here, α is the weight value for the semantic contribution of the candidate word itself and the joint contribution of the semantic cluster.

[0051] As α increases to 0.8, the retrieval system will place greater emphasis on the semantic contribution of the candidate words themselves, directly affecting the quantification accuracy of semantic irreplaceability.

[0052] Step 3.4: Select the top 3 words with the highest semantic sensitivity scores for each query term as core words, that is, select the most important words in the entire query chunk.

[0053] Step 4, optimize the branches of sparse search, such as Figure 2 As shown, an improved sparse algorithm that integrates S_sem and inverse document frequency (IDF) is used. The formula for the relevance score between a document and its core words is:

[0054] (2)

[0055] Wherein, TF(bool) represents the word frequency count value, with 0 / 1 marking whether the word appears; S_sem is the semantic sensitivity score; and IDF(w) reflects the statistical scarcity of the target word w.

[0056] The algorithm is improved to obtain scores for each word chunk for retrieval. Boolean word frequencies can be cached as a document to improve retrieval efficiency. When a core word does not appear in any word chunk, a semantic cluster is used to replace it, while S_sem is halved. Semantic clusters are introduced to supplement entities, thus replacing part of the query rewriting function. During the query process, using a large model for query rewriting is beneficial for supplementing and improving entities, but compared to semantic clusters, it takes too long, which is detrimental to later answer generation. In a test of 1000 queries, the semantic cluster method had an average execution time 51.1% lower than query rewriting (using the half-precision baichuan2 model as an example). Using semantic clusters to improve word quality increases speed while halving S_sem to reduce the impact of introduced interference. Without halving, redundant information in the semantic clusters would lead to a decrease in retrieval hit rate; if the halving ratio is too low (e.g., 1 / 3), it cannot effectively weaken interference. Therefore, halving is chosen to enhance the method's rationality.

[0057] like Figure 3 As shown, the branch optimizing dense retrieval employs a query vector enhancement strategy based on CoreScore weighting. By integrating semantic importance and statistical scarcity, it achieves precise retrieval optimization within the vector space. This method corely comprises two parts: a quantization function and a vector enhancement mechanism. The core word score, CoreScore, is generated through the quantization function, defined as CoreScore(w), with the following formula:

[0058] (3)

[0059] Where △sim(w) represents the decrease in semantic similarity between the query statement and the candidate block after the target word w is masked; 1-△sim(w) represents the semantic non-substitutability of the target word w; and log(IDF(w)+1) represents the statistical weight of low-frequency words strengthened by logarithmic transformation.

[0060] The vector augmentation mechanism is based on pre-trained word embeddings. It involves weighted superposition of core word vectors and dragging the query vector in the pre-trained vector space towards a phrase containing the core words, resulting in a weighted query vector V_q' of the original query vector V_q. The formula is as follows:

[0061] (4)

[0062] Where Corescore(w) is the quantization function; λ is an adjustable parameter for the core word weighted vector strength.

[0063] The enhanced vector V_q' retains the basic semantics of the original query. Simultaneously, through the weighted superposition of core word vectors, it drags the query vector in the pre-trained vector space towards the chunk containing the core words. When used for vector retrieval, selecting a core word weighted vector strength of 0.3 allows chunks containing the core words to be prioritized for recall due to their significantly increased cosine similarity to V_q'. This fully utilizes the semantic representation capabilities of the pre-trained vectors and highlights the retrieval value of core words through an explicit weighting mechanism, without introducing new parameters or additional training. It improves the retrieval sensitivity of low-frequency core words while maintaining computational efficiency.

[0064] Step 5: The language chunks retrieved from the two branches are reordered through the result fusion output module to obtain the language chunk that is most similar to the query language chunk in the external knowledge base.

[0065] See Figure 4 The present invention provides a semantic core weighted query device for retrieval enhancement generation, comprising one or more processors, for implementing a semantic core weighted query method for retrieval enhancement generation as described in the above embodiments.

[0066] The implementation of this invention, a semantic core weighted query device for enhanced retrieval generation, can be applied to any device with data processing capabilities, such as a computer. The device can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 4 The diagram shown is a hardware structure diagram of any device with data processing capabilities, including a semantic core weighted query device for enhanced retrieval generation according to the present invention. (Except for...) Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing-capable manufacturing process in which the device is located in the embodiment may also include other hardware depending on the actual function of the data processing-capable device, which will not be described in detail here.

[0067] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] This invention also provides a readable storage medium storing a program that, when executed by a processor, implements a semantic core weighted query method for retrieval enhancement generation as described in the above embodiments.

[0070] The readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the readable storage medium can include both internal storage units of any data processing device and external storage devices. The readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0071] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A semantic core weighted query method for enhanced retrieval generation, characterized in that, Includes the following steps: Step 1: Obtain the input Chinese query text and match, filter and sort it with the language blocks in the external knowledge base to obtain the top-ranked candidate language blocks; Step 2: Obtain the cosine similarity between candidate words and words in candidate blocks in the query text and filter accordingly to form semantic neighbors. Candidate words and the obtained semantic neighbors together constitute semantic clusters, forming a candidate word-semantic cluster mapping relationship. Step 3: Design two masking processes to calculate the semantic similarity reduction, obtain the semantic sensitivity score, filter out the core word list and its corresponding semantic sensitivity score, and select core words according to the semantic sensitivity score, including: Step 3.1: Mask only the candidate words, calculate the semantic similarity difference between the query text and the target text before and after masking, and denote it as △Sim1; Step 3.2: Mask the candidate words and their semantic clusters simultaneously, and calculate the semantic similarity difference, denoted as △Sim2; Step 3.3, define the semantic sensitivity score as S_sem, and its formula is: ， Where α is the weight value of the semantic contribution of the candidate word itself and the joint contribution of the semantic cluster; Step 3.4: Select the top N words with the highest semantic sensitivity scores for each query term as core words, that is, select the most important words in the entire query block, where N is a natural number greater than 0; Step 4: Optimize the sparse retrieval branch by using an improved sparse algorithm that integrates semantic sensitivity scores and inverse document frequency; optimize the dense retrieval branch by generating core word scores through a quantization function, and then weighting the original query vector based on the core word scores to obtain an enhanced query vector for vector retrieval. Optimized dense retrieval employs a query vector enhancement strategy based on core keyword scores. By integrating semantic importance and statistical scarcity, it achieves precise retrieval optimization within the vector space. The core of optimized dense retrieval comprises two parts: a quantization function and a vector enhancement mechanism. The quantization function is defined as CoreScore(w), and its formula is as follows: ， Where △sim(w) represents the decrease in semantic similarity between the query statement and the candidate block after the target word w is masked; 1-△sim(w) represents the semantic irreplaceability of the target word w; log(IDF(w)+1) represents the statistical weight of low-frequency words strengthened by logarithmic transformation; Step 5: The language chunks retrieved by the optimized sparse search branch and the optimized dense search branch are reordered through the result fusion output module. The language chunks that are most similar to the query language chunks are obtained from the external knowledge base, and the final results are sorted and output.

2. The semantic core weighted query method for retrieval enhancement generation according to claim 1, characterized in that, In step 1, the chunk matching and filtering are performed by semantic vector matching and keyword search, and the chunks are sorted according to their relevance.

3. The semantic core weighted query method for retrieval enhancement generation according to claim 1, characterized in that, In step 2, after calculating the cosine similarity, the top few similar words matched by each candidate word are selected based on the obtained cosine similarity. Then, words below the threshold ω are filtered out, and the retained words are taken as semantic neighbors.

4. The semantic core weighted query method for retrieval enhancement generation according to claim 1, characterized in that, In step 4, the formula for the relevance score between the document and the core keywords is: ， Wherein, TF(bool) represents the word frequency count value, with 0 / 1 marking whether the word appears; S_sem is the semantic sensitivity score; and IDF(w) reflects the statistical scarcity of the target word w.

5. The semantic core weighted query method for retrieval enhancement generation according to claim 1, characterized in that, In step 4 The vector augmentation mechanism is based on pre-trained word embeddings. It involves weighted superposition of core word vectors and dragging the query vector in the pre-trained vector space towards a phrase containing the core words, resulting in a weighted query vector V_q' of the original query vector V_q. The formula is as follows: ， Where V_w is the word vector extracted from the word embedding layer of the pre-trained model, which contains the contextual semantic information of the word, and λ is an adjustable parameter for the weighted vector strength of the core word.

6. A semantic core weighted query device for enhanced retrieval generation, characterized in that, It includes one or more processors for implementing a semantic core weighted query method for retrieval enhancement generation as described in any one of claims 1-5.

7. A readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements a semantic core weighted query method for retrieval enhancement generation as described in any one of claims 1-5.

Images