Information retrieval method and system based on large language model topology semantic reordering

Abstract

The application discloses a kind of information retrieval method and system based on large language model topological semantic reordering, including Euclidean rough screening, topological gravity calculation, power law sorting and result output step.This method uses vector embedding model to carry out rough screening to query and candidate document, then calls large language model to obtain the logarithmic probability value of each candidate document, and is re-sequenced according to power law decay law.The application also discloses a recall boundary determination method and a model alignment degree detection method based on semantic decay index.In polysemy disambiguation task, retrieval accuracy can be improved from 69% to 99%, with the advantages of high retrieval accuracy and self-adaptive model alignment state.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and information retrieval technology, and more specifically to an information retrieval method and system based on topological semantic reordering of a large language model. Background Technology

[0002] With the widespread application of large language models in natural language processing, retrieval augmentation generation technology has become an important means to improve the accuracy and timeliness of model output. Retrieval augmentation generation systems combine external knowledge bases with language models, enabling them to reference relevant documents during the response generation process, thereby reducing the generation of illusory content and providing more reliable information. Existing retrieval augmentation generation systems generally employ vector similarity-based retrieval methods, which determine the relevance ranking of query text and candidate documents by calculating the Euclidean distance between them. This geometric distance-based method implicitly assumes that semantic influence diffuses isotropically in the high-dimensional embedding space, meaning that semantic influence decays uniformly in all directions centered on the query vector.

[0003] However, this method has the following shortcomings in practical applications: First, Euclidean distance cannot effectively distinguish the semantic differences of polysemous words in different contexts. Taking the word "Apple" as an example, this word can refer to both a technology company and a fruit, and the vector similarity-based method cannot accurately determine the user's query intent based on the context, which is the semantic trap problem; Second, through experimental analysis of 2055 semantic transfer data, the inventors found that there is almost no correlation between Euclidean distance and the actual semantic transfer probability of the language model, and its regression determination coefficient is extremely low (R²). 2 = 0.008), which means that traditional vector retrieval methods are inherently unable to effectively predict the semantic understanding preferences of language models; third, there is a significant difference in semantic processing ability between models that have undergone human feedback reinforcement learning alignment and those that have not been aligned, but existing retrieval methods have failed to utilize this characteristic to further improve retrieval accuracy.

[0004] In view of the above-mentioned technical deficiencies, this application proposes an information retrieval method and system based on topological semantic reordering of large language models to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide an information retrieval method and system based on topological semantic reordering of large language models, in order to solve the technical problems in existing retrieval enhancement generation systems, such as the inability of Euclidean distance to effectively perform semantic disambiguation, insufficient ability to predict semantic transition probabilities of language models, and failure to fully utilize model alignment effects to improve retrieval accuracy.

[0006] To achieve the above objectives, the present invention provides the following technical solution: An information retrieval method based on topological semantic reordering of a large language model includes the following steps: A Euclidean coarse screening step, receiving user queries, converting the query and candidate documents into high-dimensional vector representations using a vector embedding model, calculating the Euclidean distance between the vector corresponding to the query and the vectors corresponding to each candidate document, and selecting the K closest candidate documents as the initial screening result set; a topological gravity calculation step, constructing a prompt template containing query context and candidate documents for the K candidate documents in the initial screening result set, inputting the prompt template into the large language model, and obtaining the log probability value of each candidate document being selected; a power-law sorting step, sorting the candidate documents in descending order according to the log probability values, wherein the descending order follows a power-law decay law. , where α is the decay exponent; the result output step outputs the reordered document list after the power-law sorting step.

[0007] Preferably, in the topological gravity calculation step, the prompt template adopts a multiple-choice question format, presenting each candidate document as an option to the large language model, and obtaining the log probability value of the first tag of each option by parsing the log probability field returned by the large language model.

[0008] Preferably, a recall boundary determination step is set between the Euclidean coarse screening step and the topological gravity calculation step. This recall boundary determination step determines whether the reordering depth k is greater than the preset evaluation depth. When the reordering depth k is greater than the evaluation depth, the topological gravity calculation step and the power law sorting step are executed. When the reordering depth k is not greater than the evaluation depth, the initial screening result set is directly output.

[0009] Preferably, the Euclidean distance is calculated using cosine similarity, and the dimension of the high-dimensional vector representation ranges from 768 to 4096. The decay index α ranges from 0.5 to 10.0, and the α value corresponding to the large language model that has undergone human feedback reinforcement learning alignment processing is higher than the α value corresponding to the large language model that has not undergone alignment processing.

[0010] In the above technical solution, the information retrieval method and system based on topological semantic reordering of a large language model provided by the present invention have the following beneficial effects: 1. This invention introduces a topological semantic re-ranking mechanism based on the logarithmic probability value of a large language model after Euclidean coarse screening. This mechanism leverages the implicit semantic knowledge learned by the large language model during pre-training and alignment training to construct a more accurate semantic distance metric than Euclidean distance. Experiments on 2055 semantic transfer datasets demonstrate that the regression determination coefficient between topological distance and semantic transfer probability reaches 0.393 to 0.905, while the regression determination coefficient of Euclidean distance is only 0.008. When using the Akaike information criterion for model selection, the information criterion difference between the power-law model and the exponential decay model is greater than 150, corresponding to an evidence ratio exceeding 10 to the power of 30. In a disambiguation task containing 100 polysemous queries, with a re-ranking depth of 5, this method improves the two-dimensional accuracy from 69% to 99% and the average reciprocal ranking from 0.858 to 1.0.

[0011] 2. This invention discovers and utilizes the power-law decay law in semantic propagation of large language models. The decay exponent α is not an inherent property of the model architecture but a parameter that can be adjusted through alignment training. Experiments show that when alignment behavior is suppressed, α is approximately 2.18, close to the baseline of the inverse square law in physics; after alignment via human feedback reinforcement learning, α can reach 6.60, an increase of 203%; while more advanced models exhibit an intermediate level of α, approximately 2.73, indicating that advanced alignment strategies can retain higher semantic entropy to support creative associations while maintaining response accuracy.

[0012] 3. This invention discovers and utilizes the recall boundary phenomenon, where retrieval accuracy significantly increases when the re-ranking depth exceeds the evaluation depth. At a re-ranking depth of 2, accuracy remains at 69%, but the average reciprocal rank improves from 0.858 to 0.938. At a re-ranking depth of 3, accuracy increases to 82.5%, and at a re-ranking depth of 5, accuracy further improves to 99%. Each additional re-ranking candidate adds approximately 1000 milliseconds of latency, and the system can dynamically adjust the re-ranking depth based on the latency budget of the application scenario. Attached Figure Description

[0013] Figure 1 A flowchart illustrating the information retrieval method provided in an embodiment of the present invention; Figure 2 A comparison chart of the predictive capabilities of Euclidean distance and topological distance for semantic transition probability provided in an embodiment of the present invention; Figure 3 A comparison chart of the attenuation index α under different alignment conditions provided in the embodiments of the present invention; Figure 4 This is a schematic diagram of the recall boundary phenomenon provided in an embodiment of the present invention; Figure 5 A diagram illustrating the accuracy versus delay trade-off provided for embodiments of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings. It should be noted that the large language model described in the present invention is not limited to a specific model architecture, version or commercial product. Any generative language model that can receive text input and output tokens and their corresponding log-probability is applicable to the technical solutions of the present invention.

[0015] Example 1: This invention proposes an information retrieval method based on topological semantic reordering of a large language model, comprising the following steps: S1, Euclidean coarse screening step: Receive user query, use vector embedding model to convert query and candidate documents into high-dimensional vector representations respectively, calculate Euclidean distance and output the initial screening result set.

[0016] In this embodiment, the step specifically includes: The system calls a high-dimensional vector embedding model (such as a Transformer encoder with dimensions d ∈ [768, 4096]) to combine the user query Q and the various documents D in the candidate document library. i Mapped to continuous high-dimensional real vectors respectively and .

[0017] Subsequently, the Euclidean distance between the two vectors is measured by calculating the cosine similarity, as shown in the following formula:

[0018] In the formula, d is the feature dimension of the vector. and These represent the component weights of the query vector and document vector in the j-th dimension, respectively. The system is based on... The values ​​are sorted in descending order from largest to smallest, and the first K documents are selected to form the initial screening result set.

[0019] It should be noted that the Euclidean coarse screening implicitly assumes that semantic influence diffuses isotropically in the embedding space. However, in complex semantic scenarios such as polysemous word disambiguation, the predictive determination coefficient of Euclidean distance is extremely low (R0). 2 = 0.008), therefore this step is only used to quickly narrow down the search space and reduce the latency cost of subsequent calculations.

[0020] S2, Recall Boundary Determination Step: Determine whether the reordering depth k is greater than the preset evaluation depth, and perform adaptive truncation.

[0021] In this embodiment, the step specifically includes: Define the preset evaluation depth as E (E = 2 under the first two accuracy evaluation metrics), and define the current topology reordering depth set by the system as k (k ≤ K).

[0022] When k ≤ E, the system determines that it has not exceeded the recall boundary and directly outputs the initial screening result set; when k > E, the subsequent topological gravity calculation is triggered.

[0023] It should be noted that this invention experimentally establishes the existence of a recall boundary: when the reordered candidate pool fails to cover the correct answers suppressed by Euclidean distance, blindly reordering cannot improve the final recall accuracy. Introducing this decision step allows the system to achieve a dynamic trade-off between latency budget and retrieval accuracy (i.e., Pareto front), with each additional candidate document adding approximately 1000 milliseconds of inference latency.

[0024] S3, Topological Gravity Calculation Steps: Based on the initial screening results, use a large language model to obtain the log probability value of each candidate document being selected.

[0025] In this embodiment, the step specifically includes: The top k candidate documents obtained from the initial screening are used to construct a prompt template in the form of a multiple-choice question. The prompt template includes a contextual question stem (user query Q) and a corresponding list of options (candidate documents). ).

[0026] The prompt template is input into the large language model to be evaluated, and the model's sampling temperature parameter is set to be close to 0. The system obtains the log probability value of each candidate document as the correct answer by parsing the log probability field of the first tag of each option returned by the large language model's application programming interface. .

[0027] It should be noted that this logarithmic probability value represents the topological distance between the query node and the document node within the complex cognitive network of the model. (i.e., the shortest path hop count) overcomes the technical bias that traditional geometric spaces cannot handle semantic traps.

[0028] S4, Power-law sorting steps: Sort in descending order based on logarithmic probability values, following the power-law decay rule.

[0029] In this embodiment, the step specifically includes: The system abandons the traditional exponential decay model and innovatively re-ranks documents based on the power-law decay principle. The semantic transition probability P between candidate documents and the query and their ranking position rank satisfy the following power-law relationship:

[0030] In the formula, Indicates candidate documents Based on the topological position ordinal numbers after descending log-probability sorting, α represents the semantic decay exponent. Validated using the Akaike Information Criterion, the difference ΔAIC between this power-law model and the exponential decay model is >150, and the evidence ratio exceeds 10 to the power of 30, proving that semantic influence follows the dynamic diffusion of network topology. After sorting, the system outputs the final re-ranked result list for downstream retrieval enhancement generation.

[0031] S5, Alignment Quantification Detection Step (Additional Detection Function): Evaluate the alignment status of the large language model by fitting the decay index.

[0032] In this embodiment, the system utilizes the aforementioned power law to provide a quantitative detection mechanism for the alignment degree of a large language model. The specific calculation process is as follows: In logarithmic coordinates, the logarithm of the sort position is used. The independent variable is the logarithmic value of the absolute value of the logarithmic probability output by the model. Using [variable name] as the dependent variable, perform linear regression to calculate the decay exponent α:

[0033] In the formula, n is the size of the test sample set. and These are the sample means of the independent and dependent variables, respectively.

[0034] Furthermore, this invention discloses the physical mapping relationship between the attenuation index α and the effective sampling temperature τ of the large language model:

[0035] In the formula, To suppress the physical vacuum baseline decay exponent under alignment behavior ( ≈ 2.18). This formula shows that after large language models undergo reinforcement learning alignment training with human feedback, they implicitly reduce the effective temperature τ of the model by optimizing human preference scores. For example, a model in a strongly aligned state (α ≈ 6.60) has its effective temperature compressed to about one-third of the baseline, thus producing a probabilistic focusing effect in the semantic topological space. Through this step, the present invention can numerically monitor and evaluate the safety and alignment depth of different models.

Claims

1. An information retrieval method based on topological semantic reordering of a large language model, characterized in that, Includes the following steps: Euclidean coarse screening steps: Receive user queries, use vector embedding model to convert the query and candidate documents into high-dimensional vector representations respectively, calculate the Euclidean distance between the vector corresponding to the query and the vectors corresponding to each candidate document, sort them from near to far according to the Euclidean distance, and select the top K candidate documents as the initial screening result set; Topological gravity calculation steps: For the K candidate documents in the initial screening result set, construct a prompt template containing the query context and the candidate documents, input the prompt template into the large language model, and obtain the log probability value of each candidate document being selected; Power-law sorting steps: The candidate documents are sorted in descending order based on the logarithmic probability values ​​to obtain a re-sorting result. The descending order follows a power-law decay law. , where α is the decay exponent and rank is the sorting position; Output steps: Output the reordered document list after the power-law sorting step.

2. The information retrieval method based on topological semantic reordering of a large language model according to claim 1, characterized in that, In the topological gravity calculation step, the prompt template adopts a multiple-choice question format, presenting each candidate document as an option to the large language model, and obtaining the log probability value of the first tag of each option by parsing the log probability field returned by the large language model.

3. The information retrieval method based on topological semantic reordering of a large language model according to claim 1, characterized in that, Between the Euclidean coarse screening step and the topological gravity calculation step, there is also a recall boundary determination step. The recall boundary determination step determines whether the reordering depth k is greater than the preset evaluation depth. When the reordering depth k is greater than the evaluation depth, the topological gravity calculation step and the power law sorting step are executed. When the reordering depth k is not greater than the evaluation depth, the initial screening result set is directly output.

4. The information retrieval method based on topological semantic reordering of a large language model according to claim 1, characterized in that, The Euclidean distance is calculated using cosine similarity, and the dimensions of the high-dimensional vector range from 768 to 4096. In the power-law sorting step, the decay index α ranges from 0.5 to 10.0, where the α value corresponding to the large language model that has undergone human feedback reinforcement learning alignment is higher than the α value corresponding to the large language model that has not undergone alignment.

5. An information retrieval system based on topological semantic reordering of a large language model, characterized in that, include: The Euclidean search module is configured to receive user queries, perform vector embedding and Euclidean distance calculation, and output a preliminary set of candidate documents. The topological gravity calculation module is connected to the Euclidean retrieval module and is configured to call the application programming interface of the large language model to obtain the log probability value of each candidate document in the initial screening candidate document set. The reordering module, connected to the topological gravity calculation module, is configured to sort the candidate documents in descending order according to the log probability value and the power-law decay law, and generate reordered search results. The recall boundary determination module is connected to the Euclidean retrieval module and the topological gravity calculation module, respectively. It is configured to determine whether the reordering depth exceeds a preset evaluation threshold. When the reordering depth exceeds the evaluation threshold, the topological gravity calculation module is triggered to perform reordering. Otherwise, the initial screening candidate document set is directly output.

6. The information retrieval system based on topological semantic reordering of a large language model according to claim 5, characterized in that, The topological gravity calculation module is configured to support access to large language models with logarithmic probability output capabilities, including generative language models based on the Transformer architecture. The system also includes an alignment detection module connected to the topological gravity calculation module, configured to calculate a decay exponent α by fitting the power-law relationship between the logarithmic probability value and the sorting position, and use the decay exponent α as a quantitative indicator to evaluate the alignment degree of the large language model.

7. A method for detecting the alignment degree of a large language model based on semantic decay index, characterized in that, Includes the following steps: Prompt set construction steps: Construct a prompt set containing multiple sets of semantic association tests. Each set of tests contains an anchor word and multiple candidate target words with different semantic association strengths with the anchor word. Probability acquisition steps: Input each group of tests into the large language model to be detected, obtain the log probability value of each candidate target word, and sort the candidate target words from high to low according to the log probability value to obtain the sorting position of each candidate target word; Exponential fitting steps: In a logarithmic coordinate system, perform linear regression fitting with the logarithm of the sort position as the independent variable and the logarithm of the absolute value of the log probability as the dependent variable, and use the resulting slope as the decay exponent α. Alignment determination steps: The alignment degree of the large language model is determined based on the attenuation index α, wherein α is greater than the first threshold and is determined to be in a strong alignment state, and α is less than the second threshold and is determined to be in a weak alignment state, wherein the first threshold is greater than the second threshold.

8. The method for detecting the alignment degree of a large language model based on semantic decay index according to claim 7, characterized in that, The attenuation index α and the effective temperature parameter τ of the large language model satisfy an inverse proportional relationship α=α0 / τ, where α0 is the baseline attenuation index; The model, after being aligned by reinforcement learning with human feedback, achieves semantic focus enhancement by reducing the effective temperature τ, which is manifested as an increase in the attenuation exponent α.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the information retrieval method according to any one of claims 1 to 4, or the steps of the alignment detection method according to claim 7 or 8.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the information retrieval method according to any one of claims 1 to 4, or the steps of the alignment detection method according to claim 7 or 8.

Images