Intelligent question answering method based on semantic enhancement PageRank algorithm

Abstract

The invention discloses an intelligent questioning and answering method based on a semantic enhancement PageRank algorithm, and relates to the technical field of intelligent questioning and answering. According to the method, after associated documents of questions queried by a user are screened out, the semantic relevancy between different statements in each associated document is firstly calculated, and then the semantic relevancy is applied to the PageRank algorithm, so that the questions queried by the user are obtained. By means of the semantic enhancement PageRank algorithm, statements in the associated document can be sorted, key statements strongly related to the logic intention of a user query question can be screened out, and after the key statements and the user query question are spliced, the key statements and the user query question are input into a generation model to obtain an answer result. According to the method, the improved semantic enhancement PageRank algorithm is used for carrying out text compression on the associated document, so that the burden brought by redundant information can be reduced, the reasoning speed of the generation model can be increased, interference brought by the redundant information can be reduced, and improvement of the answering precision, efficiency and robustness of intelligent question answering is facilitated.

Description

Technical Field

[0001] This application relates to the technical field of intelligent question answering, and in particular to a knowledge question answering method based on a semantic enhanced PageRank algorithm. Background Art

[0002] With the rapid development of natural language processing technology, intelligent question answering systems and dialogue systems driven by large-scale language models (such as GPT, deepseek, etc.) have received extensive attention and research in various fields. Traditional intelligent question answering systems usually adopt two main methods for information processing: one is the pure retrieval method, which relies on keyword matching or vector space models to query and retrieve answers for user queries Query. The other is to directly generate answers to user queries Query through neural network models. Both of these traditional methods have significant limitations, and the generated answers often lack accuracy. Therefore, more and more intelligent question answering systems have introduced the RAG (Retriever-Augmented Generation) framework.

[0003] The RAG technology is an AI technology that combines retrieval and generation. In the retrieval stage, it first queries an external knowledge base based on the user query Query to obtain multiple relevant documents, and then in the generation stage, it uses a large language model to generate answers based on the user query Query and the retrieved relevant documents. The RAG technology can combine the advantages of information retrieval and generation models, extract relevant documents from massive texts to generate more accurate answers compared to traditional methods, aiming to enhance the accuracy and factuality of large language models, thereby facilitating the improvement of the reliability, consistency, and answer accuracy of intelligent question answering.

[0004] However, in order to improve the accuracy and quality of the generated answers as much as possible, the relevant documents generated by the intelligent question answering system based on the RAG framework in the retrieval stage often carry relatively large amounts of data content. This not only brings a large computational burden, resulting in an impact on the inference speed, but the redundant information in the data content carried by the relevant documents will also interfere with the model inference process in the generation stage, affecting the answer efficiency and answer content accuracy of the intelligent question answering system. Summary of the Invention

[0005] In view of the above problems and technical requirements, this application proposes an intelligent question answering method based on a semantic enhanced PageRank algorithm. The technical solution of this application is as follows:

[0006] An intelligent question answering method based on a semantic enhanced PageRank algorithm, the intelligent question answering method includes:

[0007] Obtain the user's query question Query, and detect several associated documents with the highest similarity to the user's query question Query from the text database;

[0008] Calculate the semantic relevance matrix M of each associated document. The element m in the i-th row and j-th column of the semantic relevance matrix M ij represents the normalized semantic relevance between the statement s i and the statement s j in the associated document, where both i and j are integer parameters;

[0009] Apply the PageRank algorithm model and iterate and calculate in combination with the semantic relevance matrix M of the associated document until convergence, to obtain the PageRank value PR(s i ) of any statement s i in the associated document;

[0010] Based on the PageRank value PR(s i ) of any statement s i in each associated document, calculate the score Score(s i ) of the statement s i , and screen out several statements with the highest scores as the key statements of the associated document;

[0011] Concatenate the user's query question Query with the key statements of each associated document and input them into the generation model to obtain the answer result for the user's query question Query.

[0012] A further technical solution thereof is that calculating the normalized semantic relevance m i between the statement s j and the statement s ij includes:

[0013] Calculate the text content similarity ω i between the statement s j and the statement s ij , and use the positions of the statement s i and the statement s j in the associated document to correct the text content similarity ω ij and complete the normalization process to obtain the semantic relevance m ij .

[0014] A further technical solution thereof is that the text content similarity between the statement s i and the statement s j is:

[0015] ω ij = α·Sim(s i , s j ) + β·Key(si ,s j )

[0016] Among them, Sim(s i ,s j ) is the statement s i and statements j The semantic similarity of Key(s i ,s j ) is the statement s i Keywords and sentences in j The keyword similarity between the keywords in ; α and β are weighting coefficients, 0<α<1, 0<β<1 and α+β=1.

[0017] A further technical solution is to calculate the statement s i and statements j The semantic similarity Sim(s i ,s j )include:

[0018] Use the pre-trained language model BERT to convert the sentence s i Convert it into a sentence vector containing context information and use the pre-trained language model BERT to transform the sentence s j Convert to a sentence vector containing context information and calculate the sentence s i The sentence vector and sentence s j The cosine similarity between the sentence vectors of i ,s j ).

[0019] A further technical solution is to calculate the statement s i Keywords and sentences in j Keyword similarity between keywords in Key(s i ,s j )include:

[0020] Extract sentences using text feature extraction model i The set K of keywords in i , extract sentence s using text feature extraction model j The set K of keywords in j , calculate the statement s i Keywords and sentences in j The keyword similarity between the keywords in Among them, |K i ∩K j | represents the set K i With the set K j The number of keywords contained in the intersection of |K i ∩Kj | represents the set K i and the set K j The number of keywords contained in the union of

[0021] A further technical solution thereof is to use the statement s i and the statement s j The position in the associated document to correct the text content similarity ω ij and complete the normalization process to obtain the semantic relevance m ij is:

[0022]

[0023] where γ ij is the position decay factor between the statement s i and the statement s j and γ ij decays with the position spacing between the statement s i and the statement a j in the associated document; ω kj is the text content similarity between any statement s k and the statement s j in the associated document, γ kj is the position decay factor between the statement s k and the statement s j and γ kj decays with the position spacing between the statement s k and the statement s j in the associated document; n is the total number of statements in the associated document.

[0024] A further technical solution thereof is that the position decay factor γ i between the statement s j and the statement s ij =(e -λ|i-j| ) |i-j| , the position decay factor γ k between the statement s j and the statement s kj =(e -λ|k-j| ) |k-j| ; where e represents the natural logarithm base, |i - j| is the position spacing between the statement s i and the statement s j in the associated document, |k - j| is the position spacing between the statement s k and the statement s j in the associated document, and λ is a hyperparameter matching the document type of the associated document.

[0025] A further technical solution thereof is based on any statement s iThe PageRank value PR(s i ) of the calculation statement s i The score Score(s i ) includes:

[0026] Calculate the semantic relevance between the statement s i and the user query Query as the attention weight Attention(Q = s i , K = Query), and combine the attention weight and the statement s i The PageRank value PR(s i ) to obtain the score Score(s i ) = PR(s i ) · Attention(Q = s i , K = Query).

[0027] Its further technical solution is that the formula for iterative calculation by applying the PageRank algorithm model combined with the semantic relevance matrix M of associated documents is:

[0028]

[0029] Among them, PR(s i ) (t) is the PageRank value of the statement s i in the t-th round of iteration, PR(s i ) (t+1) is the PageRank value of the statement s i in the (t + 1)-th round of iteration, d i is the dynamic damping factor of the statement s i and is related to the statement importance presented by the statement features of the statement s i . The higher the statement importance of the statement s i , the larger the value of the dynamic damping factor d i of the statement s i ; M T represents the transpose of the semantic relevance matrix M, and n is the total number of statements in the associated documents.

[0030] Its further technical solution is that the dynamic damping factor d i of the statement s i is:

[0031] d i = σ(η1·Len i + η2·Ent i )

[0032] Among them, σ represents the sigmoid function, Len i represents the statement si Length, Ent i Represents statement s i Entity density in, η1 and η2 are weighting coefficients, 0 < η1 < 1, 0 < η2 < 1 and η1 + η2 = 1.

[0033] The beneficial technical effects of this application are:

[0034] This application discloses an intelligent question - answering method based on a semantic - enhanced PageRank algorithm. After screening out the relevant documents of the user's query question Query, the method further calculates the semantic relatedness between different statements in the relevant documents, and then applies the semantic relatedness to the PageRank algorithm to obtain an improved semantic - enhanced PageRank algorithm. Using the semantic - enhanced PageRank algorithm, the logical intention relatedness between the statements in the relevant documents and the user's query question Query can be sorted, so that only the key statements strongly related to the logical intention of the user's query question Query are screened out and concatenated with the user's query question Query and then input into the generation model to obtain the answer result. This method uses the improved semantic - enhanced PageRank algorithm to compress the text of the relevant documents, which can reduce the burden brought by redundant information. It can not only improve the inference speed of the generation model but also reduce the interference brought by redundant information. The practice of combining PageRank with the RAG retrieval framework significantly improves the accuracy, efficiency, and robustness of information retrieval and generation, promotes the further development of the retrieval - enhanced generation model, improves the processing speed and scalability of intelligent question - answering in large - scale knowledge - base applications, not only improves the operating efficiency of the intelligent question - answering system, but also enhances the user experience, ensures that the generated answers are more accurate and coherent, and meets the user's demand for high - quality answers.

[0035] The application of the PageRank algorithm enables the dynamic adjustment of the importance of information during the retrieval process. Aiming at the problem that traditional PageRank ignores semantic associations in text compression, this application adopts a multi - modal weight fusion mechanism to calculate the semantic relatedness between statements, thereby enhancing the system's adaptability to noise and complex contexts and being beneficial to improving the robustness of the generation model. Brief Description of the Drawings

[0036] Figure 1 It is a schematic flowchart of the intelligent question - answering method of an embodiment of this application. Detailed Embodiments

[0037] The following further describes the detailed embodiments of this application with reference to the drawings.

[0038] This application discloses an intelligent question - answering method based on a semantic - enhanced PageRank algorithm. This intelligent question - answering method can be applied to an intelligent question - answering system. Please refer toFigure 1 The flowchart shown, and this intelligent question-answering method includes the following:

[0039] Step 110, obtain the user's query question Query, and detect several associated documents with the highest similarity to the user's query question Query from the text database.

[0040] The text database stored locally includes a large number of documents and the feature vectors of each document. The feature vector of each document is obtained by encoding the document information using an embedding model. That is, the text database directly provides the feature vectors of the documents for retrieval, reducing the amount of repeated calculation. After obtaining the input user's query question Query, first use the embedding model to convert the user's query question Query into a vector representation, and then calculate the similarity between the vector representation of the user's query question Query and the feature vectors of each document in the text database, and screen out several documents with the highest similarity as the associated documents of the user's query question Query. The commonly used similarity calculation method is cosine similarity, and the number of finally screened associated documents can be set customarily. In the conventional method, directly splice the user's query question Query with the associated documents screened in this step and input them into the generation model, but in this application, further text compression is performed on each associated document through steps 120 to 140.

[0041] Step 120, calculate the normalized semantic relevance between different sentences in each associated document to obtain the semantic relevance matrix M. Any element m in the i-th row and j-th column of the obtained semantic relevance matrix M ij represents the normalized semantic relevance between sentence s i and sentence s j in the associated document, where both j and j are integer parameters, and 1 ≤ i ≤ n, 1 ≤ j ≤ n, and n is the total number of sentences in the associated document.

[0042] For any sentences s i and s j in the associated document, when calculating the semantic relevance m ij between the two sentences, first calculate the text content similarity ω i between sentence s j and sentence s ij , and then instead of directly performing normalization processing, further strengthen the document structure features, and use the positions of sentence s i and sentence s j in the associated document to correct the text content similarity ω ij and complete the normalization processing to obtain the semantic relevance m ij, thus measuring the semantic relevance from both aspects of text content and document structure to improve accuracy. The following is a separate introduction to these two aspects:

[0043] (1) Calculate the sentence s i and the sentence s j The text content similarity ω ij between them is also considered comprehensively from two aspects, and the calculation formula is:

[0044] ω ij =α·Sim(s i , s j ) + β·Key(s i , s j )

[0045] Among them, Sim(s i , s j ) is the semantic similarity between the sentence s i and the sentence s j . Key(s i , s j ) is the keyword similarity between the keywords in the sentence s i and the keywords in the sentence s j . This item can compensate for the limitations of semantic similarity and improve the accuracy of the obtained text content similarity. α and β in the above formula are weighting coefficients, 0 < α < 1, 0 < β < 1 and α + β = 1. In one embodiment, α = 0.7 and β = 0.3 are taken.

[0046] That is, the text content similarity ω ij is obtained by weighted calculation of the semantic similarity and keyword similarity between sentences. The calculation methods of these two parts are:

[0047] (a) Semantic similarity Sim(s i , s j )

[0048] First, use the pre-trained language model BERT to convert the sentence s i into a sentence vector containing context information, and use the pre-trained language model BERT to convert the sentence s j into a sentence vector containing context information. Then, calculate the cosine similarity between the sentence vector of the sentence s i and the sentence vector of the sentence s j to obtain the semantic similarity Sim(s i , s j ).

[0049] (b) Keyword similarity Key(s i , s j )

[0050] First, use the text feature extraction model to extract the set \(K\) of keywords in the statement \(s\) i , and use the text feature extraction model to extract the set \(K\) of keywords in the statement \(s\) i . The text feature extraction model used here is TF-IDF or NER. j , and use the text feature extraction model to extract the set \(K\) of keywords in the statement \(s\) j . The text feature extraction model used here is TF-IDF or NER.

[0051] Then, calculate the keyword similarity Key(\(s\) i , \(s\) j ) between the keywords in the statement \(s\) i and the keywords in the statement \(s\) j as follows:

[0052]

[0053] where \(|K\) i \(\cap K\) j | represents the number of keywords contained in the intersection of the set \(K\) i and the set \(K\) j , and \(|K\) i \(\cup K\) j | represents the number of keywords contained in the union of the set \(K\) i and the set \(K\) j .

[0054] The sentence vectors generated by the pre-trained language model BERT have the characteristic of context sensitivity. Combining with the keyword similarity for compensation can break through the limitation that the traditional cosine similarity calculation only depends on surface features, and can solve the problems of anaphora resolution (such as "he" referring to a specific person) and semantic equivalence (such as "computer" = "PC"), so that the text content similarity \(\omega\) ij calculated therefrom can more accurately represent the similarity of the text content between the statement \(s\) i and the statement \(s\) j .

[0055] (2) Use the positions of the statement \(s\) i and the statement \(s\) j in the associated document to correct the text content similarity \(\omega\) ij and complete the normalization process to obtain the semantic relatedness \(m\) ij , and the formula is:

[0056]

[0057] where \(\omega\) kj is the text content similarity between any statement \(s\) k and the statement \(s\) j in the associated document, \(\gamma\) kj is the position of the statement \(s\) kand statement s j The position attenuation factor between them is γ kj As statement s k and statement s j decays with the position spacing in the associated document. γ ij is the position attenuation factor between statement s i and statement s j The position attenuation factor between them is γ ij As statement s i and statement s j decays with the position spacing in the associated document.

[0058] The addition of the position attenuation factor can suppress the spurious correlation between distant statements, further improving the accuracy of the finally obtained semantic relevance m ij .

[0059] In one embodiment, the position attenuation factor γ i between statement s j and statement s ij =(e -λ|i-j| ) |i-j| , the position attenuation factor γ k between statement s j and statement s kj =(e -λ|k-j| ) |k-j| . Where e represents the natural logarithm base, |i - j| is the position spacing between statement s i and statement s j in the associated document, and |k - j| is the position spacing between statement s k and statement s j in the associated document. λ is a hyperparameter matching the document type of the associated document. The document types of the associated document include news and papers. When the document type of the associated document is news, λ = 0.1, and when the document type of the associated document is a paper, λ = 0.3.

[0060] Step 130, apply the PageRank algorithm model to iteratively calculate in combination with the semantic relevance matrix M of the associated document until convergence, and obtain the PageRank value PR(s i ) of any statement s i in the associated document. The traditional PageRank algorithm ignores the problem of semantic association in text compression, while the semantic enhanced PageRank algorithm improved by this application by introducing the semantic relevance matrix M can well make up for this defect. The iterative calculation formula of the improved semantic enhanced PageRank algorithm is:

[0061]

[0062] Among them, PR(si ) (t) is the PageRank value of statement s i in the t-th round of iteration, PR(s i ) (t+1) is the PageRank value of statement s i in the (t + 1)-th round of iteration, the integer parameter t ≥ 1, and the PageRank value of statement s i in the first round of iteration is obtained by initialization. M T represents the transpose of the semantic relevance matrix M.

[0063] In addition to introducing the semantic relevance matrix M, the fixed damping factor in the iterative formula of the traditional PageRank algorithm model is modified to the above dynamic damping factor d i , statement s i 's dynamic damping factor d i is related to the statement importance presented by the statement features of statement s i . The higher the statement importance of statement s i , the larger the value of the dynamic damping factor d i of statement s i , thus realizing dynamic adjustment.

[0064] In one embodiment, the calculation formula of the dynamic damping factor d i of statement s i is:

[0065] d i = σ(η1·Len i + η2·Ent i )

[0066] where σ represents the sigmoid function. Len i represents the length of statement s i , Ent i represents the entity density in statement s i . An entity refers to a real-world object or abstract concept with specific semantic meaning in a statement, such as a person's name and a place name, etc. The entity density measures the density of entities in a statement and reflects the information concentration of the text, and can be specifically carried out according to existing natural language processing methods.

[0067] The above η1 and η2 are weighting coefficients, 0 < η1 < 1, 0 < η2 < 1 and η1 + η2 = 1. The weighting coefficients η1 and η2 can be updated by the reinforcement learning policy gradient or fixed. In one embodiment, η1 = 0.7 and η2 = 0.3 are taken.

[0068] Step 140, based on any statement s in each associated document iPageRank value PR(s i )Calculation statement s i Score(s i ). In order to improve robustness, semantic PageRank and statistical features (word frequency, position) are combined to perform key ranking, including:

[0069] First, the statement s is evaluated. i The semantic relevance to the user query Query is used as the attention weight Attention (Q = s i , K = Query), and then combine the attention weight and sentence s i PageRank value PR(s i ) get statement s i Score(s i )for:

[0070] Score(s i )=PR(s i )·Attention(Q=s i ,K=Query)

[0071] Then, several sentences with the highest scores are selected as the key sentences of the associated document, thereby selecting the key sentences in the associated document that are strongly related to the logical intent of the user's query question Query (for example, if the question asks "reason", the focus is on causal sentences), and filtering out sentences with low relevance to the logical intent of the user's query question Query, which is equivalent to achieving text compression for the associated document and reducing redundant information. Text compression is completed for each associated document through steps 120 to 140, and the key sentences of each associated document are extracted.

[0072] Step 150, finally, the user query question Query and the key sentences of each related document are concatenated to form a context that is conducive to the reasoning of the generation model, and then input into the generation model to obtain the answer result for the user query question Query. In addition, in order to maintain the consistency of the context in multiple rounds of dialogue, it is necessary to update the dialogue history after each user query question Query is received. By limiting the maximum rounds of the dialogue history (for example, selecting the first 5 rounds of dialogue each time), it is ensured that the context of the generation model will not be too long, and too many historical dialogues will not affect the generation quality.

[0073] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.

Claims

1. An intelligent question-answering method based on a semantic-enhanced PageRank algorithm, characterized in that, The intelligent question-answering method includes: Obtain a user's query question Query, and detect several associated documents with the highest similarity to the user's query question Query from a text database; Calculate the semantic relevance matrix M of each associated document. The element m in the i-th row and j-th column of the semantic relevance matrix M ij represents the statement s in the said associated document i and the statement s j The normalized semantic relevance between them, where both i and j are integer parameters; Iteratively calculate the PageRank algorithm model in combination with the semantic relevance matrix M of the associated documents until convergence to obtain the PageRank value PR(s i ) of any statement s in the associated document i ); Based on any statement s in each associated document i 's PageRank value PR(s i ), calculate the score Score(s i ) of the statement s i ), and select several statements with the highest scores as the key statements of the associated document; Concatenate the user's query question Query with the key sentences of each associated document and input them into a generation model to obtain an answer result for the user's query question Query.

2. The intelligent question-answering method according to claim 1, wherein Computing statement s i and statement s j The normalized semantic relatedness m ij includes: Calculation statement s i and statement s j The text content similarity ω ij , using statement s i and statement s j The position of statement s ij in the associated document corrects the text content similarity ω ij .

3. The intelligent question-answering method according to claim 2, wherein Statement s i and statement s j The text content similarity between them is: ω ij = α·Sim(s i , s j ) + β·Key(s i , s j ) Among them, Sim(s i , s j ) is the semantic similarity between statement s i and statement s j , and Key(s i , s j ) is the keyword similarity between the keywords in statement s i and the keywords in statement s j ; α and β are weighting coefficients, where 0 < α < 1, 0 < β < 1, and α + β = 1.

4. The intelligent question-answering method according to claim 3, wherein Calculate the semantic similarity Sim(s i and statement s j , which includes: i , s j ) Convert the statement s using the pre-trained language model BERT i into a sentence vector containing context information, and convert the statement s using the pre-trained language model BERT j into a sentence vector containing context information, and calculate the cosine similarity between the sentence vector of the statement s i and the sentence vector of the statement s j to obtain the semantic similarity Sim(s i , s j ).

5. The intelligent question-answering method according to claim 3, wherein Calculation statement s i and the keywords in statement s j The keyword similarity Key(s i , s j ) includes: Extract the set K of keywords in the statement s using the text feature extraction model i from the statement s i Extract the set K of keywords in the statement s using the text feature extraction model j from the statement s j Calculate the keyword similarity between the keywords in the statement s i and the keywords in the statement s j where |K ∩K i | represents the number of keywords contained in the intersection of the set K j and the set K i and |K j ∪K i | represents the number of keywords contained in the union of the set K j and the set K i and the set K j .

6. The intelligent question and answer method according to claim 2, wherein Using statement s i and statement s j to correct the text content similarity ω ij in the associated document and complete the normalization process to obtain the semantic relevance m ij as follows: Among them, γ ij is the position attenuation factor between statement s i and statement s j and γ ij attenuates with the position spacing between statement s i and statement s j in the associated document; ω kj is the text content similarity between any statement s k and statement s j in the associated document, γ kj is the position attenuation factor between statement s k and statement s j and γ kj attenuates with the position spacing between statement s k and statement s j in the associated document; n is the total number of statements in the associated document.

7. The intelligent question-answering method according to claim 5, characterized in that Statement s i and Statement s j The position decay factor γ ij =(e -λ|i-j| ) |i-j| , Statement s k and Statement s j The position decay factor γ kj =(e -λ|k-j| ) |k-j| ; where e represents the natural base, |i - j| is the position distance between Statement s i and Statement s j in the associated document, |k - j| is the position distance between Statement s k and Statement s j in the associated document, and λ is a hyperparameter matching the document type of the associated document.

8. The intelligent question-answering method according to claim 1, characterized in that Based on any statement s in each associated document i 's PageRank value PR(s i ) to calculate the score of statement s i Score(s i ) includes: Computing statement s i The semantic relevance with the user's query problem Query is used as the attention weight Attention(Q = s i , K = Query), and combined with the attention weight and statement s i 's PageRank value PR(s i ) to obtain the score Score(s i ) = PR(s i ) · Attention(Q = s i , K = Query).

9. The intelligent question-answering method according to claim 1, wherein The formula for iterative calculation by applying the PageRank algorithm model in combination with the semantic relevance matrix M of associated documents is: Among them, PR(s i ) (t) is the PageRank value of statement s i in the t-th iteration, and PR(s i ) (t+1) is the PageRank value of statement s i in the (t + 1)-th iteration. d i is the dynamic damping factor of statement s i and is related to the statement importance presented by the statement features of statement s i . The higher the statement importance of statement s i , the larger the value of the dynamic damping factor d i of statement s i . M T represents the transpose of the semantic relevance matrix M, and n is the total number of statements in the associated documents.

10. The intelligent question-answering method according to claim 9, wherein Statement s i The dynamic damping factor d i is as follows: d i = σ(η1·Len i + η2·Ent i ) Among them, σ represents the sigmoid function, Len i represents the length of statement s i , Ent i represents the entity density in statement s i , and η1 and η2 are weighting coefficients, where 0 < η1 < 1, 0 < η2 < 1, and η1 + η2 = 1.

Images