MDS dimension reduction-based retrieval enhancement generated semantic matching optimization method and system

Abstract

The invention provides an MDS dimension reduction-based retrieval enhancement generation semantic matching optimization method and system and a storage medium. The method comprises the following steps of: obtaining a user problem and high-dimensional semantic vectors of a plurality of candidate text segments; constructing a distance matrix based on the two high-dimensional semantic vectors to obtain a high-dimensional semantic similarity; performing dimension reduction on the distance matrix by adopting a multi-dimensional scale analysis (MDS) method, and calculating low-dimensional semantic similarity between the user question and each text fragment based on a distance relationship after dimension reduction; performing weighted fusion on the low-dimensional semantic similarity and the high-dimensional semantic similarity to generate a mixed similarity; and sorting the mixed similarities from high to low, and selecting the text fragments corresponding to the top n of the mixed similarities for generating response output. By reducing the dimension of the high-dimensional retrieval vector to the low-dimensional space, the interference of irrelevant dimensions is reduced, the semantic matching precision and efficiency are improved, and then the quality of the text recalled by the large language model is improved.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and information retrieval technology, specifically to a retrieval enhancement generation semantic matching optimization method and system based on MDS dimensionality reduction. Background Technology

[0002] Retrieval-Augmented Generation (RAG) is an intelligent question-answering architecture that integrates information retrieval and Large Language Model (LLM) generation capabilities. Its core lies in how to accurately retrieve the most relevant text fragments to the user's query from a massive knowledge base and input these fragments as prompts into the large model, thereby generating more accurate and context-sensitive answers.

[0003] Existing technologies mainly employ high-dimensional vector similarity retrieval, keyword-enhanced retrieval, and re-ranking techniques to achieve the aforementioned goals. High-dimensional vector retrieval methods rely on embedding models such as mxbai-embed-large and bge-m3 to represent text as high-dimensional semantic vectors and use cosine similarity for matching; keyword-enhanced retrieval combines traditional retrieval models such as BM25, which to some extent supplements the shortcomings of vector retrieval in keyword matching; re-ranking techniques use cross-encoders to perform secondary ranking of the initially screened text fragments to further improve ranking accuracy.

[0004] However, these methods still have several limitations. First, high-dimensional semantic spaces often contain a large amount of redundancy and noise, which can easily interfere with similarity judgments, causing search results to deviate from the user's true intent. Second, general embedding models are insufficient in expressing specialized domain terminology, exhibiting weak domain adaptability in scenarios such as law, finance, and procurement, affecting the accuracy and professionalism of the answers. In addition, while methods such as re-ranking have improved performance, they have high computational costs and reduced real-time response capabilities, requiring the overall system to make a difficult trade-off between search accuracy and processing efficiency. Summary of the Invention

[0005] This invention proposes a retrieval enhancement method, system, and storage medium based on MDS dimensionality reduction to address the technical problem of low accuracy and efficiency in existing semantic matching, which leads to poor text quality in large language models.

[0006] To address the aforementioned technical problems, this invention provides a retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction, comprising the following steps: Step S1: Obtain the high-dimensional semantic vectors of the user's question and several candidate text fragments; Step S2: Construct a distance matrix based on the two high-dimensional semantic vectors to obtain the high-dimensional semantic similarity; Step S3: Use the Multidimensional Scaling Analysis (MDS) method to reduce the dimensionality of the distance matrix, and calculate the low-dimensional semantic similarity between the user question and each text fragment based on the distance relationship after dimensionality reduction; Step S4: Weightedly fuse the low-dimensional semantic similarity and the high-dimensional semantic similarity to generate a hybrid similarity; Step S5: Sort the mixed similarity from high to low, and select the text segments corresponding to the top n mixed similarity rankings to generate response output, where n is greater than or equal to 1.

[0007] Preferably, the distance matrix constructed in step S2 is weighted, and the weight of the distance item related to the user's question is set to be greater than the weight of other distance items. The expression is: ; In the formula, This represents the weighting of user questions. ; q represents the distance between vectors i and j; q represents the label of the user question vector.

[0008] Preferably, step S3 includes: Step S31: Construct a centering matrix H, and double-center the distance matrix using the centering matrix to obtain the inner product matrix; Step S32: Perform eigenvalue decomposition on the inner product matrix B, and sort the eigenvalues ​​in descending order to obtain... The projection matrix is ​​formed by the eigenvectors corresponding to the largest eigenvalues. ; Step S33: Project all original vectors onto the projection matrix. In 3D space, we obtain a low-dimensional coordinate representation; Step S34: Calculate low-dimensional semantic similarity using Euclidean distance between low-dimensional coordinates.

[0009] Preferably, a weighted centralization matrix is ​​used to replace the centralization matrix in step S1, wherein the weighted centralization matrix... The expression is: ; In the formula, for The identity matrix, Indicates the number of vectors; It is a column vector of all 1s with dimension . ; It is the user issue centralization strength parameter; Let q be the basis vector where the q-th bit is 1 and the rest are 0.

[0010] Preferably, a priority scoring method is used to replace the feature value sorting in step S32, wherein the priority scoring method... The expression is: ; ; In the formula, Indicates the first The combined eigenvalues ​​of the eigenvectors; Indicates the first eigenvectors Correlation with user question vectors; This represents the tradeoff coefficient.

[0011] Preferably, the low-dimensional semantic similarity mentioned in step S34 The expression is: ; In the formula, in the formula, Indicates the first Low-dimensional projected coordinates of a text fragment; This represents the low-dimensional projection coordinates of the user query vector after dimensionality reduction.

[0012] Preferably, the high-dimensional semantic similarity is calculated based on cosine similarity.

[0013] Preferably, the mixed similarity The calculation expression is: ; In the formula, Indicates the weighting coefficient; Indicates low-dimensional semantic similarity; This represents high-dimensional semantic similarity.

[0014] This invention also provides a retrieval enhancement and semantic matching optimization system based on MDS dimensionality reduction, applicable to the above-mentioned method, the system comprising: The vector generation module is used to obtain high-dimensional semantic vectors of user questions and several candidate text fragments; The dimensionality reduction module is used to construct a distance matrix based on the high-dimensional semantic vector, calculate the high-dimensional semantic similarity, and use the MDS method to map the vector to a low-dimensional space. The similarity calculation module is used to calculate the low-dimensional semantic similarity between the user query and each text fragment in a low-dimensional space, and to weight and fuse the low-dimensional similarity with the high-dimensional semantic similarity to generate a hybrid similarity. The relevant segment filtering module is used to select several text segments based on the mixed similarity. The generation module is used to input user questions and text fragments into a large language model and generate output content that is semantically related to the user's query.

[0015] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0016] The beneficial effects of this invention include at least the following: By introducing a semantic dimensionality reduction mechanism based on multidimensional scaling analysis (MDS), this invention effectively improves the accuracy of semantic matching in the retrieval enhancement generation system. Unlike traditional methods that rely on directly calculating similarity using high-dimensional semantic vectors, this invention maps the original vectors to a low-dimensional space, removing redundant dimensions and semantic noise. This makes the semantic structural relationship between user queries and knowledge base fragments clearer and more interpretable, thereby improving the recall quality of relevant text fragments.

[0017] During the dimensionality reduction process, as an additional technical feature, this invention designs a user-question-centric guidance strategy. Through a weighted distance matrix and a centralization transformation, the user query is placed at the core of the low-dimensional semantic space. This design can more effectively preserve semantic features related to user intent, making the matching algorithm more sensitive to key user information. It solves the problems of insufficient adaptability of general embedding models in professional fields and inaccurate similarity assessment caused by excessive semantic noise in vectors.

[0018] Furthermore, this invention integrates low-dimensional Euclidean similarity with high-dimensional cosine similarity. Through a hybrid similarity calculation mechanism, it retains the interpretability of low-dimensional data and the advantages of dimensionality reduction compression while compensating for potential information loss during dimensionality reduction. This strategy of joint high- and low-dimensional matching effectively balances system response speed and semantic matching accuracy, ensuring improved generation quality in practical applications.

[0019] In summary, this invention not only improves the retrieval accuracy and generation relevance of the RAG system in complex semantic scenarios, but also enhances the system's computational efficiency and professional adaptability, demonstrating good practical value in structured knowledge-intensive fields such as law, finance, and procurement. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the two-dimensional spatial relationship between user questions and various knowledge fragments in an embodiment of the present invention; Figure 3 This diagram illustrates the comparison of recall accuracy between the method of this invention and traditional cosine similarity calculation under the mxbai-embed-large embedding model. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0022] like Figure 1 As shown, this embodiment of the invention provides a retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction, including the following steps: Step S1: Obtain the high-dimensional semantic vectors of the user's question and several candidate text fragments.

[0023] Specifically, in this embodiment of the invention, the knowledge base is first processed, and the Markdown document is processed as follows: 1) Use structured Markdown documents as the raw data for the knowledge base.

[0024] 2) Divide it into text segments according to preset rules, such as by paragraph boundaries, chapter titles, or fixed length.

[0025] 3) Use embedding models such as mxbai-embed-large and bge-m3 to convert each text fragment into a high-dimensional vector to accurately capture the deep semantic features of the text.

[0026] 4) Associate the generated vectors with the corresponding text fragments and store them in the vector database to support fast similarity retrieval.

[0027] During the user question-and-answer processing phase, the same embedding model is used to transform the user's questions into high-dimensional vectors.

[0028] Step S2: Construct a distance matrix based on the two high-dimensional semantic vectors to obtain the high-dimensional semantic similarity.

[0029] Specifically, for indivual The 3D vector data is preprocessed to calculate the cosine distance matrix. , of which elements .remember This facilitates subsequent mixed similarity calculations. and They represent the first The and the first A high-dimensional vector, This indicates the calculation of cosine.

[0030] In this embodiment of the invention, the distance term related to the user question is given a higher weight, and the user question vector is set as follows: Weight is q represents the label of the user question vector. The improved distance matrix expression is as follows: ; in, This represents the distance between vectors i and j. This represents the weighting of user questions. The default setting is 0.8, which increases the weight of the distance related to the user's question, amplifies the distance relationship between the knowledge fragment and the user's question, and makes the question after subsequent dual centralization closer to the origin of the coordinates.

[0031] Step S3: Use the Multidimensional Scaling Analysis (MDS) method to reduce the dimensionality of the distance matrix, and calculate the low-dimensional semantic similarity between the user question and each text fragment based on the distance relationship after dimensionality reduction.

[0032] Specifically, in this embodiment, the MDS method based on user questions is used to reduce the dimensionality of the data. First, a centralized matrix is ​​constructed. Using a centralized matrix By performing double centering, we obtain the inner product matrix. This transformation makes the row sum and column sum of the inner product matrix B both zero, eliminating the origin dependency and facilitating eigenvalue decomposition. for The identity matrix; It is a column vector of all 1s, with dimension 1. ; for A matrix consisting entirely of 1s, where each element is 1; It is the distance matrix after squaring each element.

[0033] Perform eigenvalue decomposition on B. ,in It is an eigenvalue diagonal matrix. Arranged in descending order of eigenvalues.

[0034] Take before The eigenvectors corresponding to the largest eigenvalues ​​constitute the projection matrix. low-dimensional coordinates for The OK. Generally, the value is 2 or 3, which can represent high-dimensional vectors in two-dimensional or three-dimensional space.

[0035] By calculating the Euclidean distance between different coordinate points This yields the semantic similarity between knowledge fragments and user-generated questions. (Low-dimensional spatial distance) This allows for low-dimensional preservation of the original spatial distance relationships. The above operations are based on the fact that eigenvalues ​​represent the importance of data in different dimensions; larger eigenvalues ​​correspond to dimensions that can explain more data variance, meaning the data differences in that direction are more significant, while smaller eigenvalues ​​correspond to dimensions that contain less discrepancy information and can be considered noise. Large eigenvalues ​​represent the data The most important orthogonal direction can preserve the distance relationships in the original high-dimensional space to the greatest extent. If there are a large number of low-quality fragments in the knowledge base, dimensionality reduction can improve the robustness of retrieval and remove semantic noise.

[0036] Then calculate MDS similarity The specific steps are as follows: ; in, The first in the knowledge base Low-dimensional projected coordinates of a text fragment; The user queries the low-dimensional projected coordinates of the vector after dimensionality reduction; The L2 norm is the Euclidean distance, which measures the straight-line distance between two low-dimensional points.

[0037] In this embodiment, in order to improve the accuracy of matching for user problems, the following improvement method is proposed: a weighted central matrix is ​​used to replace the central matrix in the MDS, and priority scoring is used to replace feature value sorting.

[0038] Specifically, weighted centralized matrix The expression is: ; in Let be a basis vector with the q-th bit set to 1 and the rest to 0, used to mark the location of user questions. This is the user issue centralization strength parameter, default value. .

[0039] During the dual-centralization phase, by adjusting This controls the position of the user's problem in a low-dimensional space, making it closer to the origin.

[0040] During the sorting process, the feature vectors are reordered according to their relevance to the user's question, prioritizing the dimensions most relevant to the user's question. The operation is as follows: Calculate each feature vector. Correlation with user question vectors: , For the one-hot encoded vector of the user problem, , of which One bit is 1, and the rest are 0. (Comprehensive eigenvalue) and correlation Sort by dimension: in, To balance the coefficients, the top z eigenvectors, selected in descending order of priority scores, are used to construct the projection matrix. By incorporating features related to user questions as a reference benchmark for reducing dimensionality, the accuracy of matching can be further improved.

[0041] Step S4: Weighted fusion of low-dimensional semantic similarity and high-dimensional semantic similarity to generate hybrid similarity.

[0042] Specifically, in this embodiment, MDS similarity in a low-dimensional space is used. Cosine similarity in high-dimensional space , which serves as the similarity between the question vector and the knowledge base vector.

[0043] ; In the formula The value ranges from 0 to 1. When simply calculating MDS similarity... This formula can be used to select a suitable... By combining the original cosine similarity, information loss during dimensionality reduction can be avoided.

[0044] Step S5: Sort the mixed similarity from high to low, and select the text segments corresponding to the top n mixed similarity rankings to generate the response output, where n is greater than or equal to 1.

[0045] Specifically, the similarity is compared and ranked, and the most relevant knowledge fragments are recalled as the basis for the answer; the user's question and the recalled knowledge fragments are integrated into prompt words; the specified dialogue model is called to generate an answer that is strongly related to the knowledge base based on the prompts and then returned to the user.

[0046] The following is a detailed implementation process using the above method.

[0047] This process converts regulatory documents in a specified domain into Markdown files and forms a knowledge base. The knowledge base files are then segmented using specified rules; this example uses chapter-based segmentation. An embedding model such as mxbai-embed-large is used to encode the knowledge base text as a 1024-dimensional vector.

[0048] To address potential user questions related to this regulation, a test dataset of questions was constructed, as shown in Table 1. Table 1 For each question, a cosine distance matrix is ​​constructed and MDS dimensionality reduction is performed. The k-dimensional projected coordinates are stored. To visualize the k-dimensional vectors, k=2 is used in this example. This yields the two-dimensional relationship between the question and each knowledge fragment, as shown below. Figure 2 As shown.

[0049] In this embodiment, the mxbai-embed-large embedding model was selected, and the top 3 knowledge fragments were selected for recall. Figure 2 The red dot in the lower right corner represents the two-dimensional spatial region where the vector corresponding to the user's question is located, and the green dot represents the region where each knowledge fragment is located in the space. Above the knowledge fragment is the MDS similarity between the knowledge fragment and the user's question. The darker the color, the higher the similarity, and the lighter the color, the lower the similarity.

[0050] In the calculation parameters of mixed similarity Under the condition that the n green points closest to the red point are selected as the recall fragments, they are integrated to form the input of the specified large model cue words, which is used to enhance the large model's understanding of specific knowledge domains. In the mixed similarity calculation parameters... Under the given conditions, the corresponding formula can be used for calculation.

[0051] The mxbai-embed-large embedding model was used to test the question dataset. All question test datasets were processed, and the accuracy of the answers was tested. The accuracy results are shown in Table 2.

[0052] Table 2 The table shows the accuracy rate of answering questions under different numbers of knowledge fragments, and the graph is shown below. Figure 3 As shown in the figure, the changes in the accuracy of question answers using the mxbai-embed-large embedding model under different recall conditions can be observed. It can be found that the accuracy of question answers calculated using MDS for similarity is relatively higher than that calculated using traditional cosine similarity, which illustrates the advantages of the present invention under such conditions.

[0053] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; only preferred embodiments of the present invention are illustrated. The descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. As long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0054] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this invention should be determined by the appended claims.

Claims

1. A retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction, characterized in that: Includes the following steps: Step S1: Obtain the high-dimensional semantic vectors of the user's question and several candidate text fragments; Step S2: Construct a distance matrix based on the two high-dimensional semantic vectors to obtain the high-dimensional semantic similarity; Step S3: Use the Multidimensional Scaling Analysis (MDS) method to reduce the dimensionality of the distance matrix, and calculate the low-dimensional semantic similarity between the user question and each text fragment based on the distance relationship after dimensionality reduction; Step S4: Weightedly fuse the low-dimensional semantic similarity and the high-dimensional semantic similarity to generate a hybrid similarity; Step S5: Sort the mixed similarity from high to low, and select the text segments corresponding to the top n mixed similarity rankings to generate response output, where n is greater than or equal to 1.

2. The retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction according to claim 1, characterized in that: The distance matrix constructed in step S2 is weighted, and the weights of distance items related to user questions are set to be greater than the weights of other distance items. The expression is: ； In the formula, This represents the weighting of user questions. ; q represents the distance between vectors i and j; q represents the label of the user question vector.

3. The retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction according to claim 2, characterized in that: Step S3 includes: Step S31: Construct a centering matrix H, and double-center the distance matrix using the centering matrix to obtain the inner product matrix; Step S32: Perform eigenvalue decomposition on the inner product matrix B, and sort the eigenvalues ​​in descending order to obtain... The projection matrix is ​​formed by the eigenvectors corresponding to the largest eigenvalues. ; Step S33: Project all original vectors onto the projection matrix. In 3D space, we obtain a low-dimensional coordinate representation; Step S34: Calculate low-dimensional semantic similarity using Euclidean distance between low-dimensional coordinates.

4. The retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction according to claim 3, characterized in that: The weighted centralization matrix is ​​used to replace the centralization matrix in step S1. The expression is: ； In the formula, for The identity matrix, Indicates the number of vectors; It is a column vector of all 1s with dimension . ; It is the user issue centralization strength parameter; Let q be the basis vector where the q-th bit is 1 and the rest are 0.

5. The retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction according to claim 3, characterized in that: Priority scoring is used to replace the feature value sorting in step S32. The expression is: ； ； In the formula, Indicates the first The combined eigenvalues ​​of the eigenvectors; Indicates the first eigenvectors Correlation with user question vectors; This represents the tradeoff coefficient.

6. The retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction according to claim 1, characterized in that: The low-dimensional semantic similarity mentioned in step S34 The expression is: ； In the formula, Indicates the first Low-dimensional projected coordinates of a text fragment; This represents the low-dimensional projection coordinates of the user query vector after dimensionality reduction.

7. The retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction according to claim 1, characterized in that: The high-dimensional semantic similarity is calculated based on cosine similarity.

8. The retrieval enhancement and semantic matching optimization method based on MDS dimensionality reduction according to claim 1, characterized in that: The mixed similarity The calculation expression is: ； In the formula, Indicates the weighting coefficient; Indicates low-dimensional semantic similarity; This represents high-dimensional semantic similarity.

9. A retrieval enhancement and semantic matching optimization system based on MDS dimensionality reduction, applicable to the method described in any one of claims 1 to 8, the system comprising: The vector generation module is used to obtain high-dimensional semantic vectors of user questions and several candidate text fragments; The dimensionality reduction module is used to construct a distance matrix based on the high-dimensional semantic vector, calculate the high-dimensional semantic similarity, and use the MDS method to map the vector to a low-dimensional space. The similarity calculation module is used to calculate the low-dimensional semantic similarity between the user query and each text fragment in a low-dimensional space, and to weight and fuse the low-dimensional similarity with the high-dimensional semantic similarity to generate a hybrid similarity. The relevant segment filtering module is used to select several text segments based on the mixed similarity. The generation module is used to input user questions and text fragments into a large language model and generate output content that is semantically related to the user's query.

10. A computer-readable storage medium, characterized in that: The device contains a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images