Association question generation method and device, computer equipment and storage medium

Abstract

The invention relates to the field of data processing, and discloses an associated question generation method and device, a medium and equipment, and the method comprises the steps: carrying out the retrieval of a preset external knowledge base based on an original question through a mixed retriever, inputting the original question and a retrieval result into an answer generation large model, and generating an original answer and a used fragment set; on the basis of the retrieval result and the used fragment set, residual information, not quoted by the original answer, of the retrieval result is obtained through calculation; constructing an associated question to generate a cue word; inputting the associated question generation cue word into the associated question generation large model, and generating a candidate associated question set; and performing quality inspection and screening on the candidate association problem set based on a preset condition by using the general large model to generate a final association problem. By implementing the method and the device, the problems that the correlation problem generation technology is difficult to generate correlation problems with correlation, diversity and increment value and cannot meet the requirements of users on high-quality interactive experience and deep information exploration are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a related question generation method and device, a medium and equipment. BACKGROUND

[0002] Under the background of rapid development of artificial intelligence and natural language processing technology, the question and answer system has been widely applied in intelligent customer service, education consultation, information retrieval and other fields. With the continuous improvement of users' demand for interactive experience and information depth, the traditional mode of relying on single question and answer to provide fixed answers has been difficult to meet the needs of users for theme exploration and knowledge expansion. In this scenario, the related question generation technology emerged as the times require. The related question refers to the follow-up question actively pushed by the question and answer system to the user after outputting the answer to the original question, which is highly related to the semantic or theme of the original question and not directly covered by the original answer. High-quality related questions can effectively guide users to explore related themes in depth, expand the dimension of information acquisition, and at the same time improve the interaction frequency and stickiness of users and the system, becoming one of the key indicators to measure the intelligence level and user experience of the question and answer system.

[0003] Currently, the related question generation technology has formed various mainstream implementation schemes, which are applied in different scenarios, but all have significant limitations. The keyword expansion method is a widely used early related question generation method, the core logic of which is to extract the core keywords from the original question input by the user, and then generate related questions based on the keyword frequency statistics, word co-occurrence relationship or preset dictionary. However, such methods only rely on surface word matching, lack understanding of the semantic depth and context logic of the question, and the generated related questions often have repeated semantics with the original question, only slightly different in expression, which cannot provide incremental information beyond the original question category for users, and it is difficult to cover the implicit needs behind the keywords, with serious lack of diversity and exploration. The user behavior analysis method relies on user historical interaction data to recommend high-frequency related questions by analyzing user behavior characteristics such as click frequency and question frequency of different questions. This method highly depends on the accumulation and update of historical data, and for newly launched business scenarios, unpopular questions or real-time changing user needs, due to the lack of sufficient behavior data support, it cannot generate effective related questions, with poor adaptability and timeliness. The question classification method classifies the original question into a specific category or topic by constructing a preset question category system or topic label library, and then selects related questions from the question library under the same category or topic according to rules such as random selection and hotness sorting. The defect of this method is that the construction of the category system requires a large amount of manual cost, and the category division is fixed and lagging, which cannot be flexibly adapted to the subdivision of the theme; at the same time, the questions under the same category may have low correlation with the core needs of the original question, resulting in insufficient pertinence of the generated related questions.

[0004] With the development of deep learning technology, semantic analysis method has gradually become the mainstream direction of related question generation. This method uses a pre-trained semantic model to encode the semantics of the original question, converts it into a high-dimensional vector representation, and then retrieves similar semantic vector questions from a large-scale question-answer corpus as related questions. Although this method solves the problem of insufficient semantic understanding of traditional methods, it only focuses on the semantic features of the original question itself and does not combine the content of the original answer generated by the system. If the generated related question has been completely covered by the original answer, it loses the value of follow-up questions. In addition, in the multi-round dialogue scenario, the above traditional methods generally lack context coherence, and each related question generation is treated as an independent task without combining the questions, answers and user feedback in the historical dialogue. The generated related questions are often logically disconnected from the previous text and cannot form a coherent exploration path, making it difficult to meet the user's demand for in-depth exploration of the theme.

[0005] In summary, related question generation technology is difficult to generate related questions with relevance, diversity and incremental value, and cannot meet the user's demand for high-quality interactive experience and in-depth exploration of information. SUMMARY

[0006] Therefore, the present application provides a related question generation method, device, medium and equipment to solve the problem that related question generation technology is difficult to generate related questions with relevance, diversity and incremental value, and cannot meet the user's demand for high-quality interactive experience and in-depth exploration of information.

[0007] In a first aspect, the present application provides a related question generation method, which comprises: receiving an original question input by a user, wherein the original question is a to-be-answered question raised by the user in the interaction process of a question-answer system; retrieving a pre-set external knowledge base based on the original question using a hybrid retriever to obtain a retrieval result, wherein the hybrid retriever comprises a dense vector retriever and a sparse keyword retriever; inputting the original question and the retrieval result into an answer generation large model to generate an original answer and a used fragment set, wherein the used fragment set is a set composed of fragments in the retrieval result referenced by the original answer; calculating the remaining information in the retrieval result that is not referenced by the original answer based on the retrieval result and the used fragment set; constructing a related question generation prompt based on the original question, the original answer, the retrieval result, the used fragment set and the remaining information; inputting the related question generation prompt into a related question generation large model to generate a candidate related question set; performing quality inspection and screening on the candidate related question set based on a pre-set condition using a general-purpose large model to generate a final related question and output it to a client.

[0008] The application embodiment provides the association question generation method, first, through receiving the original question input by the user, and the original question is the problem to be solved proposed by the user in the interaction process of the question and answer system, the core information demand of the user is accurately captured, the user interaction scene is directly connected, it is ensured that the problem and the actual demand of the user are highly consistent, and the subsequent process is avoided due to the problem understanding deviation deviating from the user demand.Secondly, by using the mixed retriever containing the dense vector retriever and the sparse keyword retriever, the preset external knowledge base is retrieved based on the original question to obtain the retrieval result, and the advantages of the two retrievers are fully utilized.Then, the original question and the retrieval result are input into the answer generation large model, an original answer and a used fragment set composed of a retrieval result fragment cited by the original answer are generated, the powerful semantic understanding and information integration capability of the large model are relied on, the original answer that fits the original question and is complete in content is generated, the basic information demand of the user is met, the used fragment set is recorded, and the information range used in the retrieval result is clearly defined.Based on the retrieval result and the used fragment set, the remaining information in the retrieval result that is not cited by the original answer is calculated, and the potential value of the retrieval result can be fully tapped.The uncited part in the retrieval result is not useless information, but may contain supplementary content related to the original question but not embodied in the original answer, the use of retrieval resources is maximized, the generation of the associated question has more information dimensions, and the single content of the associated question is avoided due to insufficient information utilization.Furthermore, based on the original question, the original answer, the retrieval result, the used fragment set and the remaining information, the association question generation prompt word is constructed, the comprehensive and accurate input information of the association question generation large model is provided, the core demand of the user, the existing answer content, the complete retrieval resource, the information use condition and the supplementary information are integrated, the large model clearly understands the background, the boundary and the direction of the association question generation, and the generated association question deviates from the theme or repeats the existing content due to incomplete or biased input information is avoided.Then, the association question generation prompt word is input into the association question generation large model, a candidate association question set is generated, and the large model relies on the innovative generation capability to batch produce diversified association questions related to the original question.The large model can mine potential question points related to the original question from different angles and depths based on the multi-dimensional information in the prompt word, and generate a sufficient number of candidate association questions with diverse content.Finally, the general large model is used to check and screen the candidate association question set based on the preset condition, the final association question is generated and output to the client, and the quality of the output association question is ensured.The general large model can strictly check the relevance, integrity and diversity of the candidate association question according to the preset condition, eliminate the questions that do not meet the requirements, and ensure that the finally output association question not only fits the theme of the original question, but also has high information value and diversity.By implementing the present application, the problem that related problem generation technologies are difficult to generate related problems with relevance, diversity and incremental value is solved, and the problem that high-quality interactive experience and information depth exploration cannot be met is solved.

[0009] In an optional implementation, the above-mentioned retrieving a preset external knowledge base based on the original question by using the hybrid retriever to obtain a retrieval result includes: calculating the cosine similarity after converting the original question and the document segments of the external knowledge base into vectors respectively, and the dense vector retriever adopts a vector model; performing term frequency statistics and relevance scoring on the document segments of the external knowledge base based on the keywords extracted from the original question, and the sparse keyword retriever adopts a BM25 algorithm; and obtaining Top-K document segments related to the original question as the retrieval result by weighted fusion of the retrieval results of the dense vector retriever and the sparse keyword retriever, wherein K is a preset positive integer, and the value is 1-20.

[0010] In an optional implementation, the above-mentioned inputting the original question and the retrieval result into the answer generation large model to generate an original answer and a used segment set includes: splicing the original question and the retrieval result into an input text in the format of "original question: {original question content} \n reference document segment: {retrieval result content}" by RAG Pipeline when inputting, the answer generation large model adopts a Qwen-72B-Instruct model, and is deployed in a computing environment equipped with 4x A100 graphics cards; and after outputting the original answer, locating the content coinciding with the retrieval result segment in the original answer by a text matching algorithm, marking and summarizing the corresponding coinciding segment as the used segment set.

[0011] In an optional implementation, the above-mentioned calculating the remaining information of the retrieval result not referenced by the original answer based on the retrieval result and the used segment set includes: traversing each document segment in the retrieval result, and if the current document segment meets a preset condition, determining that the current document segment belongs to the used segment set, the preset condition is that the current document segment meets any one of the conditions that the length of the continuously matched characters is greater than or equal to 10 and the proportion of the length of the continuously matched characters to the total length of the document segment is greater than or equal to 30%; and extracting and integrating all document segments in the retrieval result that are not determined as the used segment set to obtain the remaining information.

[0012] In one optional implementation, the calculation of the remaining information not referenced by the original answer based on the search results and the used fragment set includes: using a pre-trained sentence-level cross-encoder model, taking the original answer and each document fragment in the search results as input, and outputting a similarity score between the original answer and each document fragment in the search results; if the similarity score between the original answer and the current document fragment is <0.85, then the current document fragment is not referenced by the original answer and is included in the remaining information candidate set; if the similarity score between the original answer and the current document fragment is ≥0.85, then the current document fragment is determined to belong to the used fragment set; and all document fragments in the remaining information candidate set are integrated to obtain the remaining information.

[0013] In one optional implementation, the above-mentioned large-scale model for generating correlation questions is any one of OpenAI GPT-4, Claude-3, and Llama-3-70B. When calling the large-scale model for generating correlation questions, the parameters are set as follows: temperature parameter T = 0.7, cumulative probability parameter top-p = 0.9, maximum number of generated tokens max_tokens = 512, and the output results are format-validated after generation.

[0014] In one optional implementation, the aforementioned general large model adopts the GPT-4Turbo model, and the aforementioned preset conditions include semantic non-coverage condition, topic association condition, and content diversity condition.

[0015] Secondly, the present invention provides a related question generation apparatus, comprising: an acquisition module for receiving an original question input by a user, wherein the original question is an unanswered question raised by the user during interaction with a question-and-answer system; a retrieval module for using a hybrid retrieval tool to search a preset external knowledge base based on the original question to obtain retrieval results, wherein the hybrid retrieval tool includes a dense vector retrieval tool and a sparse keyword retrieval tool; and an input module for inputting the original question and the retrieval results into a large-scale answer generation model to generate an original answer and a set of used fragments, wherein the set of used fragments consists of the retrieval results referenced by the original answer. The system consists of: a set of fragments; a calculation module, used to calculate the remaining information in the search results that is not referenced by the original answer; a construction module, used to construct related question generation prompts based on the original question, the original answer, the search results, the used fragment set, and the remaining information; a generation module, used to input the related question generation prompts into the related question generation model to generate a candidate related question set; and an output module, used to perform quality checks and filtering on the candidate related question set based on preset conditions using a general model, generating the final related question and outputting it to the client.

[0016] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the association problem generation method of the first aspect or any corresponding embodiment described above.

[0017] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the association problem generation method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the method for generating related questions according to an embodiment of the present invention;

[0020] Figure 2 This is a structural block diagram of an association problem generation device according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Against the backdrop of rapid development in artificial intelligence and natural language processing technologies, question-answering systems have been widely applied in various fields such as intelligent customer service, educational consulting, and information retrieval. As users' demands for interactive experiences and in-depth information continue to rise, the traditional model of providing fixed answers in a single question-and-answer session is no longer sufficient to meet users' needs for topic exploration and knowledge expansion. In this context, related question generation technology has emerged. Related questions refer to follow-up questions that the question-answering system proactively pushes to users after outputting the original question's answer; these questions are highly relevant to the semantics or topic of the original question and are not directly covered by the original answer. High-quality related questions can effectively guide users to explore related topics in depth, expand the dimensions of information acquisition, and simultaneously increase the frequency and stickiness of user interaction with the system, becoming one of the key indicators for measuring the intelligence level and user experience of a question-answering system.

[0024] Currently, several mainstream solutions for generating related questions have emerged, and while these solutions are applied in various scenarios, they all have significant limitations. Keyword expansion is an early and widely used method for generating related questions. Its core logic involves extracting core keywords from the user's original question and then generating related questions based on keyword frequency statistics, word co-occurrence relationships, or a pre-set dictionary. However, this method relies solely on surface-level word matching, lacking an understanding of the semantic depth and contextual logic of the questions. The generated related questions often semantically overlap with the original question, differing only slightly in wording. It fails to provide users with incremental information beyond the scope of the original question and struggles to cover the implicit needs behind the keywords, resulting in a severe lack of diversity and exploratory power. User behavior analysis relies on historical user interaction data, recommending frequently occurring related questions by statistically analyzing user click frequency, question popularity, and other behavioral characteristics. This method heavily depends on the accumulation and updating of historical data. For newly launched business scenarios, niche questions, or constantly changing user needs, it lacks sufficient behavioral data support, making it unable to generate effective related questions and exhibiting poor adaptability and timeliness. The question classification method constructs a pre-defined question category system or topic tag library, uses a classification model to categorize the original question into a specific category or topic, and then selects related questions from the question library under the same category or topic according to rules such as random selection or popularity sorting. The drawbacks of this method are that the construction of the category system requires a lot of manual labor, and the category division is fixed and lagging, making it unable to flexibly adapt to subdivided topics; at the same time, questions under the same category may have low relevance to the core needs of the original question, resulting in insufficient targeting of the generated related questions.

[0025] With the development of deep learning technology, semantic analysis has gradually become the mainstream approach for generating related questions. This method uses a pre-trained semantic model to semantically encode the original question, converting it into a high-dimensional vector representation, and then retrieves questions with similar semantic vectors from a large-scale question-and-answer corpus as related questions. While this approach addresses the insufficient semantic understanding of traditional methods, it only focuses on the semantic features of the original question itself, without considering the content of the original answer generated by the system. If the generated related question has already been completely covered by the original answer, it loses its value for further inquiry. Furthermore, in multi-turn dialogue scenarios, the aforementioned traditional methods generally lack contextual coherence, often treating each generation of related questions as an independent task, without considering questions, answers, and user feedback from the historical dialogue. The generated related questions are often disconnected from the preceding logic, failing to form a coherent exploration path and thus failing to meet users' needs for in-depth topic exploration.

[0026] In summary, related question generation technologies struggle to generate related questions that combine relevance, diversity, and incremental value, thus failing to meet users' demands for high-quality interactive experiences and in-depth information exploration.

[0027] The method for generating related questions provided in this invention first receives the original question input by the user, clearly identifying it as a question raised during the interaction with the question-and-answer system. This accurately captures the user's current core information needs, directly connecting to the user's interaction scenario and ensuring a high degree of alignment between the input question and the user's actual requirements. This avoids deviations in subsequent processes due to misunderstandings of the question. Second, a hybrid search engine, incorporating both dense vector search and sparse keyword search, is used to search a pre-defined external knowledge base based on the original question, yielding search results and fully leveraging the advantages of both search engines. Then, the original question and search results are input into a large-scale answer generation model, generating an original answer and a set of used fragments (fragments referenced by the original answer). This leverages the powerful semantic understanding and information integration capabilities of the large-scale model to generate a complete and relevant original answer that meets the user's basic information needs. Furthermore, by recording the set of used fragments, the scope of information already utilized in the search results is clearly defined. Finally, based on the search results and the set of used fragments, the remaining information in the search results that was not referenced by the original answer is calculated, fully exploring the potential value of the search results. The uncited portions of the search results are not useless information; rather, they may contain supplementary content related to the original question but not reflected in the original answer. Maximizing the utilization of search resources allows for more diverse information dimensions in the generation of related questions, avoiding the problem of simplistic content due to insufficient information utilization. Furthermore, by constructing related question generation prompts based on the original question, original answer, search results, used fragment sets, and remaining information, comprehensive and accurate input information is provided to the related question generation model. This integrates core user needs, existing answer content, complete search resources, information usage, and supplementary information, allowing the model to clearly understand the background, boundaries, and direction of related question generation, preventing generated related questions from deviating from the topic or repeating existing content due to incomplete or biased input information. Next, by inputting the related question generation prompts into the related question generation model, a set of candidate related questions is generated. Leveraging the model's innovative generation capabilities, diverse related questions related to the original question are produced in batches. Based on the multi-dimensional information in the prompts, the model can mine potential questions related to the original question from different angles and depths, generating a sufficient number of diverse candidate related questions. Finally, by utilizing a general large model to perform quality checks and screening on the candidate related question set based on preset conditions, the final related questions are generated and output to the client, ensuring the quality of the output related questions. The general large model can rigorously verify the relevance, completeness, and diversity of candidate related questions based on preset conditions, eliminating questions that do not meet the requirements, and ensuring that the final output related questions not only fit the original question theme but also possess high information value and diversity.By implementing this invention, the problem that related problem generation technology is unable to generate related problems that have relevance, diversity and incremental value, and cannot meet users' needs for high-quality interactive experience and in-depth information exploration is solved.

[0028] According to an embodiment of the present invention, an embodiment of a method for generating related questions is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] This embodiment provides a method for generating related questions. Figure 1 This is a flowchart of an association question generation method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0030] Step S101: Receive the original question input by the user. The original question is the question to be answered raised by the user during the interaction with the question-and-answer system.

[0031] Specifically, the receiving operation relies on the user interface layer pre-set by the question-and-answer system. This layer includes common interaction methods such as web interfaces, mobile application interfaces, and application programming interfaces (APIs), allowing users to input their original questions through any of these methods. During the receiving process, basic preprocessing operations are performed on the user input. These preprocessing operations include removing meaningless special characters from the input text, as these characters do not carry valid semantic information and may interfere with subsequent processes' understanding of the question; correcting obvious spelling errors by comparing the input text with a pre-set general dictionary or domain-specific dictionary to identify and correct misspellings and typos; and completing omitted components due to user input habits, such as supplementing abbreviated expressions into complete sentences to ensure the received original question is complete and semantically clear. After preprocessing, the processed original question is stored in a pre-set temporary data storage module. This module is used to temporarily store intermediate data during the process, providing accurate initial input information for subsequent steps. At the same time, basic interaction data such as the time of receipt of the original question and the user ID are recorded. The user ID is used to distinguish the interaction requests of different users, and the time of receipt is used for subsequent data traceability and process timeliness management to ensure that the entire interaction process is traceable.

[0032] Step S102: Using a hybrid search engine, a search is performed on a preset external knowledge base based on the original question to obtain search results. The hybrid search engine includes a dense vector search engine and a sparse keyword search engine.

[0033] Specifically, step S102 includes:

[0034] Step a1: Convert the original question and the document fragments in the external knowledge base into vectors and calculate the cosine similarity. The dense vector retrieval tool adopts a vector model, which is one of the BGE-large-zh and E5-large-v2 vector models.

[0035] Furthermore, the dense vector retrieval unit is a component of the hybrid retrieval unit. Its core function is to mine implicit semantic relationships between texts through vector transformation and similarity calculation. The vector models used are the BGE-large-zh vector model or the E5-large-v2 vector model. Both models are pre-trained vector models with strong semantic understanding capabilities, capable of mapping text information into high-dimensional dense vectors. The Wikipedia Chinese text knowledge fragments in the knowledge base snapshot are all split into independent fragments according to semantic integrity rules, ensuring that each fragment carries clear topic information. A document fragment refers to a text fragment containing specific topic information obtained by splitting a complete document from an external knowledge base. The splitting operation is performed according to preset semantic integrity rules, ensuring that each fragment carries independent and complete semantics. Cosine similarity is a calculation metric used to measure the degree of similarity between two high-dimensional vectors. Its calculation results can reflect the semantic closeness of the original question and the document fragment. The dense vector retrieval tool is launched, invoking a pre-defined vector model to perform vector transformation on the original question. This transforms the textual information of the original question into a fixed-dimensional high-dimensional dense vector. The transformation process follows the input format requirements of the vector model, preprocessing the original question text with word segmentation and encoding. Simultaneously, the same vector transformation operation is performed on all document fragments in the external knowledge base, obtaining a high-dimensional dense vector for each document fragment. All document fragment vectors are stored in a vector index library, which is used for fast storage and retrieval of vector data, improving the efficiency of subsequent similarity calculations. After vector transformation, based on the document fragment vectors in the vector index library, the cosine similarity between the original question vector and each document fragment vector is calculated. The document fragments are then sorted from highest to lowest similarity score, yielding the preliminary search results output by the dense vector retrieval tool.

[0036] Step a2: Based on the keywords extracted from the original question, perform word frequency statistics and relevance scoring on document fragments in the external knowledge base. The sparse keyword retrieval tool uses the BM25 algorithm.

[0037] Furthermore, the sparse keyword retrieval tool is another component of the hybrid retrieval tool. Its core function is to locate document fragments explicitly related to the original question through keyword matching. The BM25 algorithm used is an information retrieval ranking algorithm based on word frequency statistics, which calculates document relevance based on the frequency and distribution of keywords in documents. Keywords are words or phrases extracted from the original question text that represent the core needs of the question. The extraction process is performed using a word segmentation algorithm and a keyword weight model, selecting words with high semantic importance as keywords. Word frequency statistics refer to counting the number and frequency of keywords in each document fragment, used to initially determine the degree of association between the document fragment and the keywords. Relevance scoring refers to calculating the relevance score of each document fragment to the original question based on the word frequency statistics results and the BM25 algorithm; the higher the score, the stronger the association between the document fragment and the original question. Keyword extraction is performed on the original question. The original question text is split into independent word units using a word segmentation algorithm, and then the weight value of each word unit is calculated using a keyword weight model. Word units with weight values ​​higher than a preset threshold are selected as keywords, forming the keyword set for the original question. Subsequently, the sparse keyword retrieval tool is started, and the BM25 algorithm is called to process all document fragments in the external knowledge base. For each document fragment, the frequency and location of each keyword in the keyword set within that fragment are first counted. This information is then input into the BM25 algorithm's calculation formula, combined with preset parameters such as document length factor and keyword inverse document frequency, to calculate the relevance score between the document fragment and the original question. All document fragments are then sorted from highest to lowest relevance score to obtain the preliminary search results output by the sparse keyword search engine. This result includes the sorted document fragments and their corresponding relevance scores.

[0038] Step a3: By weighted fusion of the search results of the dense vector searcher and the sparse keyword searcher, the top-K document fragments related to the original question are obtained as search results, where K is a preset positive integer with a value of 1-20.

[0039] Furthermore, weighted fusion refers to assigning preset weights to each document fragment in the two preliminary search results, calculating a comprehensive score for each document fragment based on these weights, and then determining the final ranking fusion strategy based on the comprehensive score. The Top-K document fragments refer to selecting the top K document fragments after sorting them from highest to lowest comprehensive score, where K is a preset positive integer based on business needs and search results. The final search result is a set of document fragments with the highest relevance to the original question, obtained after weighted fusion and filtering, used to provide information support for subsequent answer generation steps. The weight allocation scheme for the two preliminary search results is determined. The weight allocation scheme is optimized based on historical search result data. By analyzing the accuracy, recall, and other indicators of the search results under different weight combinations, the weights of the preliminary results from the dense vector search engine and the sparse keyword search engine are determined. The sum of the two weights is 1 to ensure the rationality of the weighted calculation. Subsequently, document fragments in the two preliminary search results are deduplicated. If the same document fragment appears in both preliminary results, its score in both results is retained; if a document fragment appears only in one preliminary result, its score in that result is retained, and the score for the unmentioned searcher is treated as 0. For each document fragment, its score in the dense vector searcher preliminary result is multiplied by the weight of that searcher, and then its score in the sparse keyword searcher preliminary result is multiplied by the weight of that searcher. The two products are added together to obtain the comprehensive score of the document fragment. All document fragments are sorted from highest to lowest comprehensive score. If multiple document fragments have the same comprehensive score, a secondary sort is performed based on the similarity score of the dense vector searcher. After sorting, the top K document fragments are selected, and these fragments are organized into the final search results based on their comprehensive scores from highest to lowest.

[0040] Step S103: Input the original question and the search results into the answer generation model to generate the original answer and the set of used fragments. The set of used fragments is a collection of fragments from the search results that are referenced by the original answer.

[0041] Specifically, step S103 includes:

[0042] In step b1, during input, the original question and search results are concatenated into the input text using RAG Pipeline in the format "Original Question: {Original Question Content}\nReference Document Fragment: {Search Result Content}". The large model for generating the above answer uses the Qwen-72B-Instruct model and is deployed in a computing environment equipped with 4×A100 graphics cards.

[0043] Furthermore, during input, the original question and search results are concatenated into input text according to a preset format using the RAG Pipeline. The RAG Pipeline, or Retrieval Enhancement Generation Process, is a technical framework that combines retrieved external reference information with the generative model, enhancing the accuracy and richness of the generated text by introducing external knowledge. The input text refers to the structured text, after formatting, used to input into the answer generation model. Its format is fixed as "Original Question: {Original Question Content}\nReference Document Fragment: {Search Result Content}", allowing the answer generation model to clearly distinguish between the question and reference information. The Qwen-72B-Instruct model is the specific model used by the answer generation model; it is a large-scale language model with multi-turn dialogue and accurate question-answering capabilities, capable of efficiently handling complex question-answering tasks. The computing environment refers to the hardware environment for deploying the Qwen-72B-Instruct model, consisting of servers equipped with a specific number of graphics cards, providing sufficient computing resources to support the model's operation and ensuring the efficiency and stability of the model's original answer generation. The RAG Pipeline is then started, concatenating the original question and search results according to the preset formatting rules. During the concatenation process, the "Original Question" field is first clearly marked in the input text, followed by the complete, pre-processed content of the original question, ensuring clarity and unambiguity. Next, the "Reference Document Fragment" field is marked, and then the complete text of each document fragment is entered sequentially from highest to lowest relevance in the search results, with fragments separated by preset delimiters to avoid content confusion. After concatenation, the input text undergoes length and format validation to ensure the text length meets the input limits of the answer generation model and the format strictly adheres to preset specifications, with no missing fields or incorrect symbols. Simultaneously, it is confirmed that the computing environment hosting the answer generation model is functioning normally with sufficient hardware resources. The validated input text is then transmitted to the Qwen-72B-Instruct model, triggering the model's text generation process.

[0044] Step b2: After outputting the original answer, the text matching algorithm is used to locate the content in the original answer that overlaps with the search result fragments. The corresponding overlapping fragments are marked and summarized into the above-mentioned set of used fragments.

[0045] Furthermore, after outputting the original answer, a text matching algorithm is used to locate the content in the original answer that overlaps with the search result fragments. These overlapping fragments are then marked and aggregated into a set of used fragments. The text matching algorithm compares the similarity or overlap between two text segments, identifying related content through character-level, word-level, or semantic-level comparisons. Overlapping content refers to parts of the original answer that are highly consistent with the search result fragments in semantics or literal expression; this indicates that the original answer references the corresponding search result fragments. Marking involves adding a unique identifier to the fragments in the search results that overlap with the original answer. This identifier includes the fragment's location information in the search results and the start and end range of the overlapping content, facilitating subsequent traceability and management. Immediately after the large-scale answer generation model outputs the original answer, the text matching algorithm is activated to compare the original answer with each document fragment in the search results one by one. The comparison process is divided into two stages: the first stage performs literal matching, which identifies continuous text segments in the original answer that are exactly the same or highly similar to those in the document segment through character comparison, and records the position and content of these segments; the second stage performs semantic matching, which calculates the semantic similarity between the original answer and the document segment through a pre-trained semantic model, identifies content in the original answer that, although expressed differently, is semantically derived from the document segment, and supplements the corresponding document segment with tags.

[0046] Step S104: Based on the above search results and the above set of used fragments, calculate the remaining information of the above search results that were not referenced by the above original answer.

[0047] Specifically, step S104 includes:

[0048] Step c1: Iterate through each document fragment in the above search results. If the current document fragment meets the preset conditions, it is determined that the current document fragment belongs to the above used fragment set. The preset conditions are: the current document fragment meets either the condition that the length of consecutively matched characters in the above original answer is ≥10 or the length of consecutively matched characters accounts for ≥30% of the total length of the document fragment.

[0049] Furthermore, traversal refers to performing verification operations on each document fragment sequentially according to the order of the document fragments in the search results to ensure no fragments are missed. The preset condition is a specific standard used to determine whether a document fragment is referenced by the original answer, containing two parallel conditions; satisfying either one is sufficient to determine if the fragment has been used. The length of consecutively matched characters refers to the number of consecutively overlapping characters between the text in the document fragment and the text in the original answer. The ratio of the length of consecutively matched characters to the total length of the document fragment refers to the ratio of the length of consecutively matched characters to the total length of the document fragment, used to measure the proportion of content referenced in the document fragment. The current document fragment is selected sequentially from highest to lowest relevance in the search results, and compared character by character with the original answer, recording the consecutively overlapping character sequences and their corresponding lengths. For each consecutively overlapping character sequence, it is checked whether either "the length of consecutively matched characters reaches the preset value" or "the ratio of the length of consecutively matched characters to the total length of the current document fragment reaches the preset percentage" is met. If any consecutive overlapping character sequence in the current document fragment meets the conditions, the document fragment is directly determined to belong to the set of used fragments, and the position of the fragment in the search results and the specific preset conditions it meets are recorded; if none of the character sequences in the current document fragment meet the two preset conditions, it is temporarily determined to be an unused fragment and enters the subsequent further verification stage.

[0050] Step c2 involves extracting and integrating all document fragments from the search results that were not identified as part of the used fragment set, to obtain the remaining information mentioned above.

[0051] Further, all document fragments not identified as part of the used fragment set in the search results are extracted and integrated to obtain the remaining information. Document fragments not identified as part of the used fragment set refer to search result fragments that, after verification according to the preset conditions in step c1, were determined not to be referenced by the original answer. Extraction refers to the operation of filtering out the aforementioned unused fragments from the search results. Integration refers to organizing and formatting the extracted unused fragments to form a clearly structured and logically coherent information set. First, all document fragments identified as unused in step c1 are filtered from the search results, and the original number and relevance score of each fragment in the search results are recorded to facilitate tracing the fragment's source and evaluating its information value. The filtered unused fragments undergo content deduplication; completely duplicated or highly similar fragments are identified and removed through text comparison to avoid redundant information affecting subsequent use. Subsequently, the unused fragments are sorted from high to low according to their relevance scores to the original question, prioritizing fragments with high relevance. Simultaneously, the text content of each fragment is formatted, removing irrelevant special symbols, line breaks, and whitespace characters to ensure concise and consistent text expression. Finally, all unused fragments after sorting and normalization are combined sequentially to form complete residual information, which is stored in the residual information data module. An index is also established to associate the information with the original question and the search results for easy retrieval and management later.

[0052] In some embodiments, step S104 includes:

[0053] Step d1: Using a pre-trained sentence-level cross-encoder model, the original answer and each document fragment in the search results are taken as input, and the similarity score between the original answer and each document fragment in the search results is output.

[0054] Furthermore, using a pre-trained sentence-level cross-encoder model, the original answer and each document fragment in the search results are taken as input, and a similarity score is output. The sentence-level cross-encoder model is a pre-trained natural language processing model capable of deep interactive encoding of two texts, accurately calculating the semantic similarity between them. Its output is a numerical value reflecting the degree of semantic connection between the two texts. The similarity score is the output of the sentence-level cross-encoder model, used to quantify the semantic overlap between the original answer and the document fragment. A higher score indicates that the semantics of the document fragment are closer to the original answer, and the higher the likelihood of it being cited. The pre-trained sentence-level cross-encoder model is loaded, ensuring that the model parameters and weights are properly loaded and that the computing environment resources are sufficient to guarantee model efficiency. Following the order of the document fragments in the search results, "original answer + current document fragment" is sequentially taken as a set of input data, and the text is encoded according to the format required by the sentence-level cross-encoder model, converting the text into a vector form that the model can recognize. The encoded input data is fed into the model, which uses an internal cross-attention mechanism to perform deep semantic interaction analysis between the original answer and the current document fragment, calculating their semantic overlap and relevance. After the model completes its calculation, it outputs a corresponding similarity score, recording the correspondence between this score and the current document fragment. This process is repeated for all document fragments and the original answer, forming a "document fragment - similarity score" correspondence list to provide data support for subsequent fragment determination.

[0055] Step d2: If the similarity score between the original answer and the current document fragment is <0.85, then the current document fragment is not cited by the original answer and is included in the remaining information candidate set.

[0056] Furthermore, if the similarity score between the original answer and the current document fragment is lower than a preset threshold, then the current document fragment is not cited by the original answer and is included in the remaining information candidate set. The preset threshold is a similarity scoring standard used to distinguish whether a document fragment has been cited. This threshold is determined through extensive historical data verification and can balance the accuracy and completeness of the judgment. The remaining information candidate set is a collection used to temporarily store document fragments that have not been cited by the original answer, providing alternative fragments for the final determination of subsequent remaining information. The "document fragment-similarity score" correspondence list formed in step d1 is read one by one, and the similarity score of each current document fragment is compared with the preset threshold. If the similarity score of the current document fragment is lower than the preset threshold, it indicates that the fragment has a low semantic relevance to the original answer and has not been cited by the original answer. The fragment is directly included in the remaining information candidate set, and its similarity score and original number in the search results are recorded for subsequent traceability and verification. If the similarity score of the current document fragment reaches or exceeds the preset threshold, it is not included in the remaining information candidate set for the time being, and step d3 is proceeded to further determine whether it belongs to the used fragment set. During the inclusion process, the number of fragments in the remaining candidate information set is monitored in real time. If the number is too large, a preliminary screening is initiated to prioritize the retention of fragments that are highly relevant to the original question, thus ensuring the effectiveness of the candidate information set.

[0057] Step d3: If the similarity score between the original answer and the current document fragment is ≥0.85, then the current document fragment is determined to belong to the set of used fragments.

[0058] Furthermore, if the similarity score between the original answer and the current document fragment reaches or exceeds a preset threshold, the current document fragment is determined to belong to the used fragment set. The preset threshold is consistent with the threshold in step d2, serving as a unified semantic relevance judgment standard to ensure consistency in fragment judgment logic. For the current document fragment whose similarity score reaches or exceeds the preset threshold in step d1, further verification is performed through text localization to determine if it has been referenced by the original answer. A deep semantic comparison is conducted between the current document fragment and the original answer to identify the core content in the fragment that semantically overlaps with the original answer, confirming whether the overlapping content belongs to the key information part of the original answer. If there is core semantic overlap, the document fragment is formally determined to belong to the used fragment set, and the similarity score of the fragment, the position of the core overlapping content in the original answer, and the fragment's number in the search results are recorded. If, through semantic comparison, it is found that although the similarity score reaches the preset threshold, the overlapping content between the fragment and the original answer is only an irrelevant general expression, it is re-determined as an unused fragment and included in the remaining information candidate set. After all the judgments are completed, the set of used segments is sorted from high to low according to the similarity score, which facilitates cross-validation with the search results later.

[0059] Step d4: Integrate all document fragments in the remaining information candidate set to obtain the aforementioned remaining information.

[0060] Furthermore, all document fragments in the candidate set of remaining information are integrated to obtain the remaining information. The candidate set of remaining information is a collection of document fragments temporarily stored after filtering in steps d2 and d3 that were not referenced by the original answer. Integration refers to optimizing the content, logically sorting, and standardizing the format of the fragments in the candidate set to form the final usable remaining information. First, all document fragments in the candidate set of remaining information are content-verified, eliminating fragments with incomplete content, semantic ambiguity, or irrelevant information to ensure the completeness and validity of the fragment information. Then, a semantic clustering algorithm is used to classify the fragments in the candidate set by topic, grouping fragments with similar topics into the same category, facilitating the extraction of information by topic when generating related questions. For each topic category, fragments are sorted from high to low according to their relevance score to the original question, prioritizing the presentation of core information. Simultaneously, the text content of each fragment is standardized, unifying the font format, punctuation rules, and removing irrelevant whitespace and special symbols to ensure concise and consistent text expression. Finally, all the fragments after classification, sorting and normalization are combined in sequence according to topic categories to form the remaining information with a clear structure and logical coherence. This information is stored in the remaining information data module and associated with the original question and search results, providing high-quality information input for the construction of prompt words for subsequent related questions.

[0061] Step S105: Based on the original question, the original answer, the search results, the set of used fragments, and the remaining information, construct related question-generating prompts.

[0062] Furthermore, a fixed framework for generating prompts for related questions is established. This framework comprises three core modules: system instructions, core information input, and output format requirements, ensuring a clear and logically coherent prompt structure. The system instructions module clearly defines the task positioning, generation standards, and constraints of the large-scale related question generation model. For example, it requires generating related questions that are complementary to the original answer, non-repetitive, and relevant to the original question's theme, and limits the number and expression specifications of the generated questions. In the core information input module, the original question, original answer, search results, used fragment set, and remaining information are entered sequentially according to a preset order. Each information module is clearly labeled, for example, using tags such as "[Original Question]" and "[Original Answer]" to distinguish different information types. Necessary explanatory supplements are provided for each information content—a brief overview of the information coverage and core theme of the search results, an explanation of the relationship between the used fragment set and the original answer, and annotations indicating the attributes of the remaining information as incremental supplementary content, helping the model quickly understand the meaning and function of each information module. The output format requirements module clearly specifies the output format of the related questions, such as requiring presentation in a specified structured format, ensuring that the model's output facilitates subsequent processing. After filling in the content of each module, the entire set of prompt words is text-validated to check for missing information, ambiguity, or formatting errors. For example, it verifies whether the original answer is complete, whether the remaining information accurately corresponds to the unused parts of the search results, and whether the output format is clear and unambiguous. Once the validation is passed, the final set of related question generation prompt words is generated and stored in a dedicated prompt word storage module, preparing it for subsequent input into the larger related question generation model.

[0063] Step S106: Input the above-mentioned related question generation prompt words into the related question generation model to generate a candidate related question set.

[0064] In some optional embodiments, the aforementioned large-scale model for generating association questions can be any one of OpenAI GPT-4, Claude-3, and Llama-3-70B. When calling the large-scale model for generating association questions, it supports integration through the OpenAI unified format interface. The parameters set are: temperature parameter T = 0.7, cumulative probability parameter top-p = 0.9, and maximum number of generated tokens max_tokens = 8096. After generation, the output results are format-validated to ensure interface compatibility and the integrity of the output content.

[0065] Specifically, a connection is established with the large-scale model for generating related questions through a pre-defined model call interface. The connection process must adhere to the communication protocol of the model interface to ensure the stability and security of data transmission. After establishing the connection, the related question generation prompts are converted into a text format recognizable by the model, according to the input format required by the model. If the prompts contain special characters or format deviations, preprocessing is required to eliminate format conflicts. After the processed prompts are transmitted to the large-scale model for generating related questions, the model's question generation process is triggered. The model uses semantic analysis algorithms to parse the core theme of the original question, the coverage of the original answer, and the incremental content in the remaining information from the prompts. Combining its pre-trained language knowledge and question generation logic, it mines potential related questions from different dimensions (such as supplementing details, extending scenarios, and related concepts). During the model's question generation process, the generation progress is monitored in real time to avoid interruptions due to model malfunctions. After the model has completed generation, all related questions output by the model are received. These questions are organized into a candidate related question set according to the generation order. The generation time and model confidence level of each question are initially recorded (if the model provides this parameter). The set is then temporarily stored in the candidate question storage module to prepare for subsequent format verification and quality screening steps. For the large-scale language model used to generate related questions, one of the following three models can be selected: OpenAI GPT-4, Claude-3, and Llama-3-70B. OpenAI GPT-4 is a large-scale language model with strong multimodal understanding and complex text generation capabilities, excelling at handling high-precision natural language generation tasks. Claude-3 is a large-scale language model with long text processing and precise instruction following capabilities, demonstrating outstanding performance in structured output and logical coherence. Llama-3-70B is a large-scale language model with efficient knowledge retrieval and multi-scenario adaptability, maintaining stable output quality across diverse tasks. When calling the large-scale model to generate association questions, three core parameters need to be set: The temperature parameter T controls the randomness of the generated text; a higher parameter value results in greater diversity of generated content, while a lower parameter value results in greater certainty. The cumulative probability parameter top-p controls the range of words selected during generation. By limiting the cumulative probability threshold, candidate words whose total probability reaches that threshold are selected, balancing the richness and rationality of the generated content. The maximum number of generated tokens, max_tokens, limits the total length of the model's output text, ensuring that the set of candidate association questions generated is within a preset length range, avoiding reduced processing efficiency due to excessively long outputs. Based on the document fragment similarity score returned by the retrieval layer, n fragments with high similarity ranking and high information gain (n is adaptively adjusted according to the large-scale model context length threshold) can be dynamically selected to avoid triggering large-scale model context exceptions, while retaining the core supplementary content in the remaining information. First, the corresponding association questions are selected to generate the large-scale model according to the needs of the application scenario (such as the emphasis on generation speed, diversity, and accuracy).If the scenario prioritizes the innovation and diversity of generated questions, OpenAI GPT-4 can be used; if the scenario emphasizes the structured and logically rigorous nature of the question expression, Claude-3 can be used; if the scenario has requirements for model deployment efficiency and resource consumption, Llama-3-70B can be used. After determining the model, configure the core parameters through the model call interface, setting the temperature parameter, cumulative probability parameter, and maximum number of generated tokens to preset values ​​to ensure that the parameter configuration meets the requirements for generating related questions—avoiding questions deviating from the topic due to excessively high temperature parameters, and avoiding a lack of diversity due to excessively low temperature parameters; ensuring the rationality of generated vocabulary through cumulative probability parameters, and controlling the output scale through the maximum number of generated tokens. After the parameters are configured, input the related question generation prompts into the model and start the generation process. After the model outputs the candidate related question set, perform format validation on the output results. The validation process first checks whether the set conforms to the preset output structure requirements (e.g., whether it is presented in the specified list format or set format). If the model output is unstructured text (e.g., pure paragraphs), the format is deemed non-compliant. Secondly, it checks whether each related question is a complete interrogative sentence, and whether there are any incomplete expressions (e.g., missing interrogative words, unfinished sentences) or grammatical errors (e.g., improper collocation, logical inconsistencies). Finally, it checks whether the number of questions in the set is within a reasonable range (avoiding both insufficient numbers leading to a lack of candidates in subsequent screening and excessive numbers increasing screening costs). If validation finds non-compliant formatting, incomplete question expressions, or grammatical errors, the model parameters need to be readjusted (e.g., fine-tuning temperature parameters or supplementing format instructions in the prompts), and the generated prompts for the related questions are re-entered into the model for secondary generation. If validation passes, the candidate related question set is retained and proceeds to the subsequent quality check steps.

[0066] Step S107: Use a general large model to perform quality checks and screening on the above candidate related question set based on preset conditions, generate the final related questions and output them to the client.

[0067] In some alternative embodiments, the general large model described above adopts the GPT-4Turbo model, and the preset conditions include semantic non-coverage condition, topic association condition, and content diversity condition.

[0068] Specifically, a general-purpose large-scale model is used to perform quality checks and screening on a set of candidate related questions based on preset conditions, generating the final related questions and outputting them to the client. The general-purpose large-scale model is a pre-trained large-scale language model with multi-dimensional text quality assessment capabilities, capable of accurately judging indicators such as text relevance, completeness, and diversity. The preset conditions are a pre-defined set of standards used to measure the quality of related questions, defining the core requirements that related questions must meet. The set of candidate related questions is a collection of multiple potential related questions output by the related question generation model, which must undergo quality checks and screening to determine the final usable questions. The final related questions are those that have passed the quality check, meet all preset conditions, and possess high informational and user interaction value. The client is the terminal carrier for users to interact with the question-answering system, including web interfaces, mobile application interfaces, etc., used to present the final related questions to users. First, the set of candidate related questions and the preset conditions are input into the general-purpose large-scale model. The input process must organize the data according to the input specifications of the general-purpose large-scale model, clearly distinguishing between the candidate question list and the preset condition descriptions, ensuring that the model clearly understands the inspection objectives and standards. After receiving the data, the general model performs a quality check on each related question in the candidate related question set, judging whether the question meets the requirements according to various preset criteria. Related questions that meet all preset criteria are marked, and those that do not meet the criteria are recorded for specific reasons (such as being off-topic, repeating content with the answer, or semantically redundant with other questions). After checking all questions, all marked related questions are selected. If the number of qualified questions does not reach a preset reasonable range (e.g., too few to meet user exploration needs), the process returns to the related question generation stage. The related question generation prompts or model parameters are adjusted, and a new set of candidate related questions is generated, and the quality check process is performed again. If the number of qualified questions meets the requirements, these qualified questions are compiled into the final related questions. Subsequently, the final related questions are transmitted to the client through a preset result output interface. The transmission process must use an encryption protocol to ensure data security, and the transmission status is monitored to avoid data loss or transmission delay. After receiving the final related questions, the client presents them to the user according to a preset display format (e.g., sorted by the degree of relevance to the original question), allowing the user to perform subsequent interactive operations (e.g., clicking to view the answer to a related question). The general-purpose large-scale model adopts the GPT-4Turbo model. The GPT-4Turbo model is a pre-trained large-scale language model with efficient text understanding and evaluation capabilities. It supports long text processing and complex logical judgments, accurately identifies whether related questions meet preset quality standards, and maintains stable evaluation performance across multiple scenarios. Preset conditions include semantic non-coverage, topic relevance, and content diversity.Among them, the semantic non-coverage condition means that the core semantics of the related question are not covered by the original answer, ensuring that the related question has supplementary information value and avoiding users obtaining duplicate content; the topical relevance condition means that the core topic of the related question is consistent with that of the original question, ensuring that the related question revolves around the user's initial needs and does not deviate from the core area of ​​user interest; the content diversity condition means that the semantic differences between different related questions meet the preset requirements, ensuring that the related questions can provide users with exploration directions from multiple perspectives and avoid redundancy in question content.

[0069] The method for generating related questions provided in this invention first receives the original question input by the user, clearly identifying it as a question raised during the interaction with the question-and-answer system. This accurately captures the user's current core information needs, directly connecting to the user's interaction scenario and ensuring a high degree of alignment between the input question and the user's actual requirements. This avoids deviations in subsequent processes due to misunderstandings of the question. Second, a hybrid search engine, incorporating both dense vector search and sparse keyword search, is used to search a pre-defined external knowledge base based on the original question, yielding search results and fully leveraging the advantages of both search engines. Then, the original question and search results are input into a large-scale answer generation model, generating an original answer and a set of used fragments (fragments referenced by the original answer). This leverages the powerful semantic understanding and information integration capabilities of the large-scale model to generate a complete and relevant original answer that meets the user's basic information needs. Furthermore, by recording the set of used fragments, the scope of information already utilized in the search results is clearly defined. Finally, based on the search results and the set of used fragments, the remaining information in the search results that was not referenced by the original answer is calculated, fully exploring the potential value of the search results. The uncited portions of the search results are not useless information; rather, they may contain supplementary content related to the original question but not reflected in the original answer. Maximizing the utilization of search resources allows for more diverse information dimensions in the generation of related questions, avoiding the problem of simplistic content due to insufficient information utilization. Furthermore, by constructing related question generation prompts based on the original question, original answer, search results, used fragment sets, and remaining information, comprehensive and accurate input information is provided to the related question generation model. This integrates core user needs, existing answer content, complete search resources, information usage, and supplementary information, allowing the model to clearly understand the background, boundaries, and direction of related question generation, preventing generated related questions from deviating from the topic or repeating existing content due to incomplete or biased input information. Next, by inputting the related question generation prompts into the related question generation model, a set of candidate related questions is generated. Leveraging the model's innovative generation capabilities, diverse related questions related to the original question are produced in batches. Based on the multi-dimensional information in the prompts, the model can mine potential questions related to the original question from different angles and depths, generating a sufficient number of diverse candidate related questions. Finally, by utilizing a general large model to perform quality checks and screening on the candidate related question set based on preset conditions, the final related questions are generated and output to the client, ensuring the quality of the output related questions. The general large model can rigorously verify the relevance, completeness, and diversity of candidate related questions based on preset conditions, eliminating questions that do not meet the requirements, and ensuring that the final output related questions not only fit the original question theme but also possess high information value and diversity.By implementing this invention, the problem that related problem generation technology is unable to generate related problems that have relevance, diversity and incremental value, and cannot meet users' needs for high-quality interactive experience and in-depth information exploration is solved.

[0070] This embodiment also provides an associated question generation device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0071] This embodiment provides a device for generating related questions, such as... Figure 2 As shown, it includes:

[0072] The acquisition module 201 is used to receive the original question input by the user, which is the question to be answered raised by the user during the interaction with the question-and-answer system.

[0073] The retrieval module 202 is used to use a hybrid retrieval tool to search a preset external knowledge base based on the original question mentioned above, and obtain retrieval results. The hybrid retrieval tool includes a dense vector retrieval tool and a sparse keyword retrieval tool.

[0074] Input module 203 is used to input the original question and the search results into the answer generation model to generate the original answer and the set of used fragments, wherein the set of used fragments is a set of fragments in the search results referenced by the original answer.

[0075] Calculation module 204 is used to calculate the remaining information of the above search results that are not referenced by the above original answer based on the above search results and the above used fragment set;

[0076] Module 205 is used to construct related question-generating prompts based on the original question, the original answer, the search results, the used fragment set, and the remaining information.

[0077] The generation module 206 is used to input the above-mentioned related question generation prompt words into the related question generation model to generate a set of candidate related questions;

[0078] The output module 207 is used to perform quality checks and screening on the above candidate related question set based on preset conditions using a general large model, generate the final related questions and output them to the client.

[0079] The associated question generation device provided in this invention first receives the original question input by the user, clearly identifying it as a question raised by the user during interaction with the question-and-answer system. This accurately captures the user's core information needs, directly connecting to the user's interaction scenario and ensuring a high degree of alignment between the input question and the user's actual requirements. This avoids deviations in subsequent processes due to misunderstandings of the question. Second, it utilizes a hybrid retrieval system containing both dense vector retrieval and sparse keyword retrieval to search a pre-defined external knowledge base based on the original question, fully leveraging the advantages of both retrieval systems. Then, by inputting the original question and search results into a large-scale answer generation model, it generates the original answer and a set of used fragments (fragments referenced by the original answer). This leverages the powerful semantic understanding and information integration capabilities of the large-scale model to generate a complete and relevant original answer that meets the user's basic information needs. Furthermore, by recording the set of used fragments, it clearly defines the scope of information already utilized in the search results. Finally, based on the search results and the set of used fragments, it calculates the remaining information in the search results that was not referenced by the original answer, fully exploring the potential value of the search results. The uncited portions of the search results are not useless information; rather, they may contain supplementary content related to the original question but not reflected in the original answer. Maximizing the utilization of search resources allows for more diverse information dimensions in the generation of related questions, avoiding the problem of simplistic content due to insufficient information utilization. Furthermore, by constructing related question generation prompts based on the original question, original answer, search results, used fragment sets, and remaining information, comprehensive and accurate input information is provided to the related question generation model. This integrates core user needs, existing answer content, complete search resources, information usage, and supplementary information, allowing the model to clearly understand the background, boundaries, and direction of related question generation, preventing generated related questions from deviating from the topic or repeating existing content due to incomplete or biased input information. Next, by inputting the related question generation prompts into the related question generation model, a set of candidate related questions is generated. Leveraging the model's innovative generation capabilities, diverse related questions related to the original question are produced in batches. Based on the multi-dimensional information in the prompts, the model can mine potential questions related to the original question from different angles and depths, generating a sufficient number of diverse candidate related questions. Finally, by utilizing a general large model to perform quality checks and screening on the candidate related question set based on preset conditions, the final related questions are generated and output to the client, ensuring the quality of the output related questions. The general large model can rigorously verify the relevance, completeness, and diversity of candidate related questions based on preset conditions, eliminating questions that do not meet the requirements, and ensuring that the final output related questions not only fit the original question theme but also possess high information value and diversity.By implementing this invention, the problem that related problem generation technology is unable to generate related problems that have relevance, diversity and incremental value, and cannot meet users' needs for high-quality interactive experience and in-depth information exploration is solved.

[0080] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0081] In this embodiment, the verification device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0082] This invention also provides a computer device having the above-described features. Figure 2 The apparatus shown.

[0083] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 3 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 3 Take a processor 10 as an example.

[0084] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0085] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0086] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0087] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0088] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0089] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded via a network and stored on a remote storage medium or a non-transitory machine-readable storage medium and to be stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0090] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for generating related questions, characterized in that, The method includes: Receive the original question input by the user, which is the question to be answered raised by the user during the interaction with the question-and-answer system; The hybrid search engine is used to search a preset external knowledge base based on the original question to obtain search results. The hybrid search engine includes a dense vector search engine and a sparse keyword search engine. The original question and the search results are input into the answer generation model to generate an original answer and a set of used fragments, wherein the set of used fragments is a collection of fragments from the search results referenced by the original answer; Based on the search results and the set of used fragments, the remaining information in the search results that was not referenced by the original answer is calculated; Based on the original question, the original answer, the search results, the set of used fragments, and the remaining information, related question prompts are constructed. The prompt words generated by the related questions are input into the large model for generating related questions to generate a set of candidate related questions. The candidate related questions are quality checked and filtered based on preset conditions using a general large model, and the final related questions are generated and output to the client.

2. The method according to claim 1, characterized in that, The process of using a hybrid search engine to search a preset external knowledge base based on the original question to obtain search results includes: The original question and document fragments in the external knowledge base are converted into vectors respectively, and then the cosine similarity is calculated. The dense vector retrieval machine adopts a vector model. Based on the keywords extracted from the original question, word frequency statistics and relevance scoring are performed on document fragments in the external knowledge base. The sparse keyword retrieval tool adopts the BM25 algorithm. By weighted fusion of the search results of the dense vector searcher and the sparse keyword searcher, the top-K document fragments related to the original question are obtained as search results, where K is a preset positive integer with a value of 1-20.

3. The method according to claim 2, characterized in that, The step of inputting the original question and the search results into the answer generation model to generate the original answer and the set of used fragments includes: During input, the original question and the search results are concatenated into the input text using the RAG Pipeline in the format "Original Question: {Original Question Content}\nReference Document Fragment: {Search Result Content}". The large-scale model for generating the answer adopts the Qwen-72B-Instruct model and is deployed in a computing environment equipped with 4×A100 graphics cards. After outputting the original answer, the text matching algorithm is used to locate the content in the original answer that overlaps with the search result fragments. The corresponding overlapping fragments are marked and summarized into the set of used fragments.

4. The method according to claim 3, characterized in that, The calculation of remaining information in the search results that was not referenced by the original answer, based on the search results and the set of used fragments, includes: Traverse each document fragment in the search results. If the current document fragment meets the preset conditions, then determine that the current document fragment belongs to the set of used fragments. The preset conditions are: the current document fragment meets either the condition that the length of consecutively matched characters in the original answer is ≥10 or the length of consecutively matched characters accounts for ≥30% of the total length of the document fragment. Extract and integrate all document fragments from the search results that were not identified as part of the used fragment set to obtain the remaining information.

5. The method according to claim 3, characterized in that, The calculation of remaining information in the search results that was not referenced by the original answer, based on the search results and the set of used fragments, includes: Using a pre-trained sentence-level cross-encoder model, the original answer and each document fragment in the retrieval results are taken as input, and the similarity score between the original answer and each document fragment in the retrieval results is output. If the similarity score between the original answer and the current document fragment is <0.85, then the current document fragment is not referenced by the original answer and is included in the remaining information candidate set; If the similarity score between the original answer and the current document fragment is ≥0.85, then the current document fragment is determined to belong to the set of used fragments; The remaining information is obtained by integrating all document fragments in the candidate set of remaining information.

6. The method according to claim 4 or 5, characterized in that, The large-scale model for generating association questions can be any one of OpenAIGPT-4, Claude-3, and Llama-3-70B. When calling the large-scale model for generating association questions, the parameters set are: temperature parameter T = 0.7, cumulative probability parameter top-p = 0.9, maximum number of generated tokens max_tokens = 512, and the output results are format-validated after generation.

7. The method according to claim 6, characterized in that, The general large model adopts the GPT-4Turbo model, and the preset conditions include semantic non-coverage condition, topic association condition, and content diversity condition.

8. A device for generating related questions, characterized in that, The device includes: The acquisition module is used to receive the original question input by the user, which is the question to be answered raised by the user during the interaction with the question-and-answer system; The retrieval module is used to use a hybrid retrieval tool to search a preset external knowledge base based on the original question and obtain retrieval results. The hybrid retrieval tool includes a dense vector retrieval tool and a sparse keyword retrieval tool. The input module is used to input the original question and the search results into the answer generation model to generate the original answer and the set of used fragments, wherein the set of used fragments is a collection of fragments in the search results referenced by the original answer; The calculation module is used to calculate the remaining information of the search results that were not referenced by the original answer, based on the search results and the set of used fragments. The construction module is used to construct related question-generating prompt words based on the original question, the original answer, the search results, the set of used fragments, and the remaining information; The generation module is used to input the associated question generation prompts into the associated question generation model to generate a set of candidate associated questions; The output module is used to perform quality checks and screening on the candidate set of related questions based on preset conditions using a general large model, generate the final related questions, and output them to the client.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the association problem generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the association problem generation method according to any one of claims 1 to 7.

Images