Intelligent question and answer implementation method and system based on hybrid retrieval and deep sorting optimization

Abstract

The invention discloses an intelligent question and answer implementation method and system based on hybrid retrieval and deep sorting optimization, and relates to the technical field of information retrieval and natural language processing. Comprising the steps of 1, recalling candidate contexts highly related to user query information from a large-scale knowledge base by utilizing a mixed retrieval model; step 2, sorting the candidate contexts by using a deep sorting model; step 3, sending user query information and a context list into a large language model, preprocessing the context list by using the large language model, and outputting answer content of which the content structure and the expression mode accord with the reading habit and the cognitive mode of human beings.

Description

Technical Field

[0001] The invention discloses an intelligent question-answering implementation method and system based on hybrid retrieval and deep sorting optimization, and relates to the technical field of information retrieval and natural language processing. Background Art

[0002] With the widespread application of large language models in natural language processing and multimodal fields, existing retrieval technologies usually rely on pure vector retrieval. Although this method can efficiently capture relevant information, it often leads to redundancy of irrelevant information due to the lack of an effective sorting mechanism, thereby reducing the matching degree between retrieval results and user needs. In addition, when dealing with complex or professional information, pure vector retrieval often fails to fully grasp the complexity of retrieval. Summary of the invention

[0003] In view of the problems in the prior art, the present invention provides an intelligent question answering implementation method and system based on hybrid retrieval and deep sorting optimization, which can effectively improve the accuracy of retrieval and the overall performance of the system, and greatly enhance the accuracy and practicality of large language models in practical applications.

[0004] The specific scheme proposed by the present invention is:

[0005] The present invention also provides an intelligent question-answering implementation method based on hybrid retrieval and deep sorting optimization, comprising:

[0006] Step 1: Use a hybrid retrieval model to recall candidate contexts that are highly relevant to the user's query information from a large-scale knowledge base:

[0007] Step 11: Perform keyword matching: The hybrid retrieval model uses the BM25 algorithm and the TextRank algorithm to calculate the relevance between the user query and the text in the large-scale knowledge base.

[0008] Step 12: The hybrid retrieval model uses the word vector model to convert the user query information and the text in the large-scale knowledge base into vector form, and calculates the similarity between the vectors to screen out candidate contexts that are highly relevant to the user query information;

[0009] Step 2: Rank candidate contexts using a deep ranking model:

[0010] Step 21: Generate semantic vectors: Use the Transformer model and text convolutional neural network to generate semantic vectors of user query information and candidate context respectively.

[0011] Step 22: Integrate semantic vectors: Integrate the semantic vectors of the user query information and the candidate context to form a semantic synthesis vector.

[0012] Step 23: Calculate similarity: Calculate the similarity between the user query information and the semantic comprehensive vector of the candidate context, sort the candidate contexts, and obtain a context list;

[0013] Step 3: Send the user query information and context list to the large language model, use the large language model to preprocess the context list, and output the answer content whose content structure and expression are in line with human reading habits and cognitive patterns.

[0014] Furthermore, in step 12 of the intelligent question answering implementation method based on hybrid retrieval and deep ranking optimization, the word vector model is trained, including:

[0015] Data preprocessing: Data cleaning: remove unnecessary elements in the text,

[0016] Sentence segmentation: Segment the text by punctuation marks to generate a series of candidate sentences. Set a maximum sentence length threshold and use a greedy strategy to combine the sentences into a set of sentences with characters not exceeding the length threshold, ensuring that the generated vector can accurately capture the text semantics.

[0017] Incremental training: Continuously update model parameters to adapt the model to data in a specific field.

[0018] Furthermore, in step 21 of the intelligent question-answering implementation method based on hybrid retrieval and deep sorting optimization, the multi-head attention mechanism in the Transformer model is used to divide it into multi-head self-attention, each self-attention head focuses on different parts of the input text, and captures multiple different contextual relationships at the same time. The position encoding in the Transformer model is used to add a fixed position information to each position in the input sequence to understand the relative or absolute position of the word, and the encoder in the Transformer model is used to map the input sequence to a higher dimensional space. The decoder is used to generate a target sequence based on the output of the encoder, and the encoder and decoder are stacked by multiple identical layers, each of which contains self-attention and feedforward neural network components.

[0019] Furthermore, in step 3 of the intelligent question answering implementation method based on hybrid retrieval and deep ranking optimization, the context list is preprocessed using a large language model, including:

[0020] Eliminate redundant information: remove unnecessary information and reduce noise interference.

[0021] Correct grammatical errors: Correct grammatical errors.

[0022] Standardize professional terms and abbreviations: standardize the use of professional terms and abbreviations,

[0023] Adapt to the predetermined sequence length and vocabulary encoding standard: adjust the text to the sequence length required by the large language model and perform vocabulary encoding.

[0024] The present invention also provides an intelligent question-answering implementation system based on hybrid retrieval and deep sorting optimization, comprising a hybrid retrieval management module, a sorting management module and a large model question-answering module.

[0025] The retrieval management module uses a hybrid retrieval model to recall candidate contexts that are highly relevant to user query information from a large-scale knowledge base:

[0026] Step 11: Perform keyword matching: The hybrid retrieval model uses the BM25 algorithm and the TextRank algorithm to calculate the relevance between the user query and the text in the large-scale knowledge base.

[0027] Step 12: The hybrid retrieval model uses the word vector model to convert the user query information and the text in the large-scale knowledge base into vector form, and calculates the similarity between the vectors to screen out candidate contexts that are highly relevant to the user query information;

[0028] The ranking management module uses a deep ranking model to rank candidate contexts:

[0029] Step 21: Generate semantic vectors: Use the Transformer model and text convolutional neural network to generate semantic vectors of user query information and candidate context respectively.

[0030] Step 22: Integrate semantic vectors: Integrate the semantic vectors of the user query information and the candidate context to form a semantic synthesis vector.

[0031] Step 23: Calculate similarity: Calculate the similarity between the user query information and the semantic comprehensive vector of the candidate context, sort the candidate contexts, and obtain a context list;

[0032] The big model question-answering module feeds user query information and context lists into the big language model, uses the big language model to preprocess the context list, and outputs answer content whose content structure and expression methods conform to human reading habits and cognitive patterns.

[0033] Furthermore, when the retrieval management module of the intelligent question-answering implementation system based on hybrid retrieval and deep ranking optimization executes step 12, the word vector model is trained, including:

[0034] Data preprocessing: Data cleaning: remove unnecessary elements in the text,

[0035] Sentence segmentation: Segment the text by punctuation marks to generate a series of candidate sentences. Set a maximum sentence length threshold and use a greedy strategy to combine the sentences into a set of sentences with characters not exceeding the length threshold, ensuring that the generated vector can accurately capture the text semantics.

[0036] Incremental training: Continuously update model parameters to adapt the model to data in a specific field.

[0037] Furthermore, when the sorting management module of the intelligent question-answering implementation system based on hybrid retrieval and deep sorting optimization executes step 21, it uses the multi-head attention mechanism in the Transformer model to be divided into multi-head self-attention, and each self-attention head focuses on different parts of the input text, capturing multiple different contextual relationships at the same time, and uses the position encoding in the Transformer model to add a fixed position information to each position in the input sequence to understand the relative or absolute position of the word, and uses the encoder in the Transformer model to map the input sequence to a higher dimensional space, and uses the decoder to generate a target sequence based on the output of the encoder, and the encoder and decoder are stacked by multiple identical layers, each layer containing self-attention and feedforward neural network components.

[0038] Furthermore, the large model question answering module of the intelligent question answering implementation system based on hybrid retrieval and deep ranking optimization uses a large language model to preprocess the context list, including:

[0039] Eliminate redundant information: remove unnecessary information and reduce noise interference.

[0040] Correct grammatical errors: Correct grammatical errors.

[0041] Standardize professional terms and abbreviations: standardize the use of professional terms and abbreviations,

[0042] Adapt to the predetermined sequence length and vocabulary encoding standard: adjust the text to the sequence length required by the large language model and perform vocabulary encoding.

[0043] The benefits of the present invention are:

[0044] Improve retrieval accuracy: By combining keyword matching and vector representation technology, the system can more comprehensively and accurately recall candidate contexts related to user queries, thereby improving the accuracy of retrieval results.

[0045] Enhanced result relevance: The deep ranking model combines the Transformer model and the text convolutional neural network to perform multi-level ranking optimization on the candidate contexts, ensuring that the most relevant contexts are ranked first, thereby enhancing the relevance of the results.

[0046] Improve answer quality: The large language model question-answering module combines efficiently sorted high-quality context with user queries to generate more accurate and reliable answers, significantly improving the quality of answers.

[0047] Optimize user experience: In the pre-output processing stage, the grammatical structure of the output results is improved, the logical relationships are sorted out, and the expression style is simplified to ensure that the content structure and expression methods conform to human reading habits and improve the user experience.

[0048] Accelerate the decision-making process: The system can quickly provide accurate information to help decision makers quickly understand the information, thereby accelerating the decision-making process and improving decision-making efficiency.

[0049] Strong adaptability: Through customized training and multi-path recall strategies, the system can adapt to the knowledge search needs of various complex and professional fields, and has high flexibility and adaptability.

[0050] Reduce manual intervention: Automated processing reduces the need for manual screening and organization of information, reduces labor costs and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the application framework of the method of the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0053] Example 1

[0054] The present invention also provides an intelligent question-answering implementation method based on hybrid retrieval and deep sorting optimization, comprising:

[0055] Step 1: Use a hybrid retrieval model to recall candidate contexts that are highly relevant to the user's query information from a large-scale knowledge base:

[0056] Step 11: Perform keyword matching: The BM25 algorithm and TextRank algorithm are used in a hybrid retrieval model to calculate the relevance between the user query and the text in the large-scale knowledge base.

[0057] The BM25 algorithm is a text retrieval algorithm that evaluates the relevance of a query to a document by calculating factors such as the frequency of the query term in the document and the length of the document. The BM25 algorithm takes into account the frequency of the query term in the document, the length of the document, and the inverse document frequency of the query term in the entire document collection. The BM25 algorithm can quickly locate key information while ensuring the speed of information processing.

[0058] The TextRank algorithm is a graph-based algorithm used to evaluate the importance of words in a text. By constructing a co-occurrence graph between words and iteratively calculating the importance score of each word, the TextRank algorithm is particularly suitable for extracting key information from a text and can provide an instant and comprehensive overview of the information.

[0059] Through the BM24 and TextRank algorithms, candidate contexts that are highly relevant to user queries can be effectively screened out. The two algorithms complement each other, with the BM25 algorithm ensuring the speed of information processing and the TextRank algorithm providing a comprehensive information overview, which together improve the accuracy and recall of retrieval.

[0060] Step 12: The hybrid retrieval model uses the word vector model to convert the user query information and the text in the large-scale knowledge base into vector form, and calculates the similarity between the vectors to screen out candidate contexts that are highly relevant to the user query information.

[0061] The word vector model is also trained, including:

[0062] Data preprocessing: Data cleaning: remove unnecessary elements in the text,

[0063] Sentence segmentation: Segment the text by punctuation marks to generate a series of candidate sentences. Set a maximum sentence length threshold and use a greedy strategy to combine the sentences into a set of sentences with characters not exceeding the length threshold, ensuring that the generated vector can accurately capture the text semantics.

[0064] Incremental training: Follow the Word2Vec model guidance to continuously update model parameters to adapt the model to data in a specific field.

[0065] Step 2: Rank candidate contexts using a deep ranking model:

[0066] Step 21: Generate semantic vectors: Use the Transformer model and text convolutional neural network to generate semantic vectors of user query information and candidate context respectively.

[0067] The Transformer model is used to capture long-term dependencies, while the text convolutional neural network is used to extract local features. In this way, the model can not only understand the long-distance semantic associations between queries and documents, but also accurately capture short-distance semantic information, thereby fully understanding the multi-level meaning of the text.

[0068] Features of the Transformer model include:

[0069] Self-Attention Mechanism: The self-attention mechanism allows the model to take into account the information of all other positions in the sequence when processing each position in the sequence. This means that the model can assign different weights to different positions in the sequence as needed, thereby capturing long-term dependencies more effectively.

[0070] Through self-attention, the model can process input sequences in parallel, greatly improving the training speed, especially the performance on long sequences. Multi-Head Attention: The multi-head attention mechanism is an extension of the self-attention mechanism, which divides the self-attention into multiple heads, each of which can focus on different parts of the input. It allows the model to capture multiple different contextual relationships at the same time, enhancing the model's understanding ability.

[0071] Positional Encoding: Since the self-attention mechanism is not sequential, Transformer introduces positional encoding to add a fixed position information to each position in the input sequence, and the model can also understand the relative or absolute position of words.

[0072] Encoder-Decoder Architecture: Transformer usually adopts an encoder-decoder architecture, where the encoder maps the input sequence to a higher dimensional space, and the decoder generates the target sequence based on the encoder's output. Both the encoder and decoder are stacked with multiple identical layers, each containing self-attention and feed-forward neural network components. Feed-Forward Neural Networks: Applied independently to each position. This structure is simple and easy to parallelize.

[0073] The text convolutional neural network is a text processing model based on convolutional neural networks. It extracts local features through multi-scale convolution kernels. The design of the text convolutional neural network is relatively simple, using only one convolution and pooling layer, which can reduce the complexity of the model and speed up the training process.

[0074] Step 22: Integrate semantic vectors: Integrate the semantic vectors of the user query information and the candidate context to form a semantic synthesis vector.

[0075] Step 23: Calculate similarity: Calculate the similarity between the user query information and the semantic comprehensive vector of the candidate context, sort the candidate contexts, and obtain a context list.

[0076] Step 3: Send the user query information and context list to the large language model, use the large language model to preprocess the context list, and output the answer content whose content structure and expression are in line with human reading habits and cognitive patterns.

[0077] In step 3, the context list is preprocessed using a large language model, including:

[0078] Eliminate redundant information: remove unnecessary information and reduce noise interference.

[0079] Correct grammatical errors: Correct grammatical errors.

[0080] Standardize professional terms and abbreviations: standardize the use of professional terms and abbreviations,

[0081] Adapt to the predetermined sequence length and vocabulary encoding standard: adjust the text to the sequence length required by the large language model and perform vocabulary encoding.

[0082] The output pre-processing stage optimizes the output results of the large language model to ensure that its content structure and expression methods conform to human reading habits and cognitive patterns. Specific optimization measures include:

[0083] Improve the grammatical structure: correct grammatical errors in the output results to make them more fluent and easy to understand. Sort out the logical relationship: clearly display the logical relationship between information to help users better understand and grasp the key points of the information. Simplify the expression style: simplify the expression style, remove redundant information, make the output results more concise and clear, and facilitate quick reading and understanding.

[0084] Example 2

[0085] The present invention also provides an intelligent question-answering implementation system based on hybrid retrieval and deep sorting optimization, comprising a hybrid retrieval management module, a sorting management module and a large model question-answering module.

[0086] The retrieval management module uses a hybrid retrieval model to recall candidate contexts that are highly relevant to user query information from a large-scale knowledge base:

[0087] Step 11: Perform keyword matching: The hybrid retrieval model uses the BM25 algorithm and the TextRank algorithm to calculate the relevance between the user query and the text in the large-scale knowledge base.

[0088] Step 12: The hybrid retrieval model uses the word vector model to convert the user query information and the text in the large-scale knowledge base into vector form, and calculates the similarity between the vectors to screen out candidate contexts that are highly relevant to the user query information;

[0089] The ranking management module uses a deep ranking model to rank candidate contexts:

[0090] Step 21: Generate semantic vectors: Use the Transformer model and text convolutional neural network to generate semantic vectors of user query information and candidate context respectively.

[0091] Step 22: Integrate semantic vectors: Integrate the semantic vectors of the user query information and the candidate context to form a semantic synthesis vector.

[0092] Step 23: Calculate similarity: Calculate the similarity between the user query information and the semantic comprehensive vector of the candidate context, sort the candidate contexts, and obtain a context list;

[0093] The big model question-answering module feeds user query information and context lists into the big language model, uses the big language model to preprocess the context list, and outputs answer content whose content structure and expression methods conform to human reading habits and cognitive patterns.

[0094] As the information interaction and execution process between the modules in the above-mentioned system are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0095] Likewise, the benefits of the system of the present invention are:

[0096] Improve retrieval accuracy: By combining keyword matching and vector representation technology, the system can more comprehensively and accurately recall candidate contexts related to user queries, thereby improving the accuracy of retrieval results.

[0097] Enhanced result relevance: The deep ranking model combines the Transformer model and the text convolutional neural network to perform multi-level ranking optimization on the candidate contexts, ensuring that the most relevant contexts are ranked first, thereby enhancing the relevance of the results.

[0098] Improve answer quality: The large language model question-answering module combines efficiently sorted high-quality context with user queries to generate more accurate and reliable answers, significantly improving the quality of answers.

[0099] Optimize user experience: In the pre-output processing stage, the grammatical structure of the output results is improved, the logical relationships are sorted out, and the expression style is simplified to ensure that the content structure and expression methods conform to human reading habits and improve the user experience.

[0100] Accelerate the decision-making process: The system can quickly provide accurate information to help decision makers quickly understand the information, thereby accelerating the decision-making process and improving decision-making efficiency.

[0101] Strong adaptability: Through customized training and multi-path recall strategies, the system can adapt to the knowledge search needs of various complex and professional fields, and has high flexibility and adaptability.

[0102] Reduce manual intervention: Automated processing reduces the need for manual screening and organization of information, reduces labor costs and improves work efficiency.

[0103] It should be noted that not all steps and modules in the above-mentioned processes and system structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0104] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for implementing intelligent question answering based on hybrid retrieval and deep sorting optimization, characterized by: include: Step 1: Use a hybrid retrieval model to recall candidate contexts that are highly relevant to the user's query information from a large-scale knowledge base: Step 11: Perform keyword matching: The hybrid retrieval model uses the BM25 algorithm and the TextRank algorithm to calculate the relevance between the user query and the text in the large-scale knowledge base. Step 12: The hybrid retrieval model uses the word vector model to convert the user query information and the text in the large-scale knowledge base into vector form, and calculates the similarity between the vectors to screen out candidate contexts that are highly relevant to the user query information; Step 2: Rank candidate contexts using a deep ranking model: Step 21: Generate semantic vectors: Use the Transformer model and text convolutional neural network to generate semantic vectors of user query information and candidate context respectively. Step 22: Integrate semantic vectors: Integrate the semantic vectors of the user query information and the candidate context to form a semantic synthesis vector. Step 23: Calculate similarity: Calculate the similarity between the user query information and the semantic comprehensive vector of the candidate context, sort the candidate contexts, and obtain a context list; Step 3: Send the user query information and context list to the large language model, use the large language model to preprocess the context list, and output the answer content whose content structure and expression are in line with human reading habits and cognitive patterns.

2. According to claim 1, a method for implementing intelligent question answering based on hybrid retrieval and deep sorting optimization is characterized by: In step 12, the word vector model is also trained, including: Data preprocessing: Data cleaning: remove unnecessary elements in the text, Sentence segmentation: Segment the text by punctuation marks to generate a series of candidate sentences. Set a maximum sentence length threshold and use a greedy strategy to combine the sentences into a set of sentences with characters not exceeding the length threshold to ensure that the generated vector can accurately capture the text semantics. Incremental training: Continuously update model parameters to adapt the model to data in a specific field.

3. According to claim 1, a method for implementing intelligent question answering based on hybrid retrieval and deep sorting optimization is characterized by: In step 21, the multi-head attention mechanism in the Transformer model is divided into multi-head self-attention. Each self-attention head focuses on different parts of the input text and captures multiple different contextual relationships at the same time. The position encoding in the Transformer model is used to add a fixed position information to each position in the input sequence to understand the relative or absolute position of the word. The encoder in the Transformer model is used to map the input sequence to a higher-dimensional space. The decoder is used to generate a target sequence based on the output of the encoder. The encoder and decoder are stacked by multiple identical layers, each of which contains self-attention and feedforward neural network components.

4. The intelligent question answering method based on hybrid retrieval and deep sorting optimization according to claim 1, characterized in that In step 3, the context list is preprocessed using a large language model, including: Eliminate redundant information: remove unnecessary information and reduce noise interference. Correct grammatical errors: Correct grammatical errors. Standardize professional terms and abbreviations: standardize the use of professional terms and abbreviations, Adapt to the predetermined sequence length and vocabulary encoding standard: adjust the text to the sequence length required by the large language model and perform vocabulary encoding.

5. An intelligent question-answering implementation system based on hybrid retrieval and deep sorting optimization, characterized by It includes hybrid retrieval management module, ranking management module and large model question and answer module. The retrieval management module uses a hybrid retrieval model to recall candidate contexts that are highly relevant to user query information from a large-scale knowledge base: Step 11: Perform keyword matching: The hybrid retrieval model uses the BM25 algorithm and the TextRank algorithm to calculate the relevance between the user query and the text in the large-scale knowledge base. Step 12: The hybrid retrieval model uses the word vector model to convert the user query information and the text in the large-scale knowledge base into vector form, and calculates the similarity between the vectors to screen out candidate contexts that are highly relevant to the user query information; The ranking management module uses a deep ranking model to rank candidate contexts: Step 21: Generate semantic vectors: Use the Transformer model and text convolutional neural network to generate semantic vectors of user query information and candidate context respectively. Step 22: Integrate semantic vectors: Integrate the semantic vectors of the user query information and the candidate context to form a semantic synthesis vector. Step 23: Calculate similarity: Calculate the similarity between the user query information and the semantic comprehensive vector of the candidate context, sort the candidate contexts, and obtain a context list; The big model question-answering module feeds user query information and context lists into the big language model, uses the big language model to preprocess the context list, and outputs answer content whose content structure and expression methods conform to human reading habits and cognitive patterns.

6. The intelligent question-answering implementation system based on hybrid retrieval and deep sorting optimization according to claim 5, characterized in that When the retrieval management module executes step 12, it also trains the word vector model, including: Data preprocessing: Data cleaning: remove unnecessary elements in the text, Sentence segmentation: Segment the text by punctuation marks to generate a series of candidate sentences. Set a maximum sentence length threshold and use a greedy strategy to combine the sentences into a set of sentences with characters not exceeding the length threshold to ensure that the generated vector can accurately capture the text semantics. Incremental training: Continuously update model parameters to adapt the model to data in a specific field.

7. According to claim 5, the intelligent question-answering implementation system based on hybrid retrieval and deep sorting optimization is characterized in that the sorting When the management module executes step 21, the multi-head attention mechanism in the Transformer model is used to divide it into multi-head self-attention. Each self-attention head focuses on different parts of the input text and captures multiple different contextual relationships at the same time. The position encoding in the Transformer model is used to add a fixed position information to each position in the input sequence to understand the relative or absolute position of the word. The encoder in the Transformer model is used to map the input sequence to a higher dimensional space. The decoder is used to generate a target sequence based on the output of the encoder, and the encoder and decoder are stacked by multiple identical layers, each of which contains self-attention and feedforward neural network components.

8. The intelligent question-answering implementation system based on hybrid retrieval and deep sorting optimization according to claim 5, characterized in that The large model question answering module uses a large language model to preprocess the context list, including: Eliminate redundant information: remove unnecessary information and reduce noise interference. Correct grammatical errors: Correct grammatical errors. Standardize professional terms and abbreviations: standardize the use of professional terms and abbreviations, Adapt to the predetermined sequence length and vocabulary encoding standard: adjust the text to the sequence length required by the large language model and perform vocabulary encoding.

Images