College policy question and answer large model combined with retrieval enhancement generation technology and construction method of college policy question and answer large model

Abstract

The invention discloses a college policy question and answer large model combined with a retrieval enhancement generation technology and a construction method. The method comprises the following steps: inputting a user question large language model and question-answer text vector data about college policies; multi-level indexes are adopted to retrieve question-answer text vector data, and policy content positioning related to the user question is achieved according to a retrieval result; semantic sorting is conducted on the retrieval results according to the scores, secondary evaluation of the policy content is achieved, and the policy content with the high semantic relevancy is preferentially provided; strategy content reasoning is carried out in a large and small model combination mode, including problem rewriting and keyword extraction by using the small model and strategy content generation by using the large model, so that computing resources are optimized and time delay is reduced; and generating answers to the user questions according to content reasoning, and outputting accurate college policy-related answers. The invention aims to improve the accuracy and efficiency of college policy information retrieval.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a large language model of university policies combined with retrieval enhancement technology and a construction method thereof. Background Art

[0002] In today's university environment, faculty and students generally face challenges when searching for policy information. Traditional retrieval methods rely on web searches, which are not only time-consuming but often fail to directly organize and present web content, making it difficult for faculty and students to quickly find the information they need. This not only increases query time costs but also reduces work efficiency.

[0003] The related technologies of the present invention include:

[0004] 1. Large Language Model (LLM): First, due to limitations in training data, LLMs can produce inaccurate responses when processing unseen knowledge or information, a phenomenon known as "model hallucination." Second, LLMs' knowledge bases are typically closed at a certain point in time, making them inadequate for providing up-to-date information. Furthermore, for universities processing sensitive data, using commercial LLM services may raise privacy and data security concerns.

[0005] 2. Retrieval-Augmented Generation (RAG) technology can overcome the limitations of traditional information retrieval systems. RAG significantly improves the ability to handle complex queries by integrating the retrieval and generation stages. In the retrieval stage, RAG uses deep learning technology to retrieve relevant information from a large number of documents; in the generation stage, RAG combines the question with the search content and then generates accurate answers through LLM. This two-stage approach not only improves the accuracy of the answers, but also enhances the LLM model's ability to understand the context. RAG systems are usually combined with vector databases to store massive amounts of document information. As the name suggests, vector databases use vectors to represent data to support efficient similarity searches. Compared with traditional keyword-based search methods, vector databases can more accurately capture subtle differences between data, thereby providing more precise search results. This is particularly important for processing complex queries that require extensive contextual understanding.

[0006] Therefore, it is necessary to design a large model for university policy question answering that combines retrieval enhancement generation technology to solve the technical problems existing in the above-mentioned existing technologies. Summary of the Invention

[0007] In order to solve the technical problems of low answer accuracy and low efficiency of existing large language models in the field of university policy question answering, the present invention aims to propose a large model and construction method for university policy question answering that combines retrieval enhancement generation technology. Through an efficient retrieval mechanism and advanced vectorization technology, it achieves rapid response and accurate matching to user queries.

[0008] The present invention is achieved by utilizing the following technical solutions:

[0009] In the first aspect, the present invention proposes a large-scale university policy question-answering model that combines retrieval-enhanced generation technology, including an input module, a positioning module, an evaluation module, a reasoning module, and a question-answering generation module, wherein:

[0010] The input module inputs user questions and question-answer text vector data about university policies;

[0011] The positioning module extracts keywords based on the user question language model, uses a multi-level index to search the question-answer text vector data based on the keywords, and locates the policy content related to the user question based on the search results;

[0012] The evaluation module calculates the semantic relevance score of each document of the policy content, and semantically sorts the search results according to the score to achieve a secondary evaluation of the policy content, giving priority to providing policy content with high semantic relevance;

[0013] The reasoning module uses a combination of large and small models to perform policy content reasoning, including using a small model to rewrite user questions and extract keywords, where question rewriting rewrites user questions based on the contextual conversation history to supplement contextual information; and uses a large model to summarize the detected content and user questions obtained by the positioning module and the evaluation module for reasoning, thereby optimizing computing resources and reducing latency.

[0014] The generation module generates answers to the user questions based on content reasoning and outputs accurate answers about university policies.

[0015] In some embodiments, the vector knowledge base includes a constructed customized data set of university policies, which at least covers question-answer text vector data including education policy, scientific research policy and student affairs, and sets a multi-level index for the vector question-answer text vector data, including at least keyword index, semantic index and ontology index.

[0016] In some embodiments, the multi-level index construction further comprises the following steps:

[0017] Step 1: Use a large language model to extract question keywords and calculate corresponding cluster labels to classify and organize questions;

[0018] Step 2: Closely integrate clustering labels with university document content to build a multi-level, structured index system to improve information retrieval efficiency;

[0019] Step 3: Build a vector database and use advanced vector search to achieve efficient similarity search and information retrieval functions;

[0020] Step 4: Implement document preprocessing steps, including at least cleaning, word segmentation, and stop word removal, to improve the quality and accuracy of the index;

[0021] Step 5: Develop a semantic understanding module, using natural language processing technology to enhance the semantic relevance of the index and improve the accuracy of search results;

[0022] Step 6: Implement a dynamic index update mechanism to ensure that the index can reflect updates and changes to policy documents in real time and maintain the timeliness of information;

[0023] Step 7: Design an index optimization algorithm and regularly evaluate and optimize the index structure to ensure the efficiency and accuracy of the index;

[0024] Step 8: Build a comprehensive security mechanism to protect sensitive information from being leaked and ensure the security of the indexing system;

[0025] Step 9: Design an extensible architecture to facilitate the subsequent addition of new index levels or types to improve the flexibility and scalability of the system;

[0026] Step 10: Develop a user-defined index function to allow users to create personalized indexes based on their own needs, thus improving the user experience.

[0027] Step 11: Build a backup and recovery system to ensure that index data can be restored in time in the event of an accident to prevent data loss or damage;

[0028] Step 12: Design a strict access control mechanism to ensure that only authorized users can access sensitive or restricted index information to ensure information security;

[0029] Step 13: Develop a semantic similarity evaluation tool to evaluate the semantic relationship between different documents and provide users with richer retrieval results and relevance analysis.

[0030] In some embodiments, the positioning module and the evaluation module further include the following steps:

[0031] Step 1: Receive a query request input by a user;

[0032] Step 2: Process the user input to extract query intent and key information;

[0033] Step 3: Analyze the context of the query request to more accurately understand the user's intent;

[0034] Step 4: Rewrite the question based on the extracted query intent and key information;

[0035] Step 5: Convert the user’s natural language query into a machine-understandable vector representation;

[0036] Step 6: Implement query intent classification to classify queries into predefined categories, such as vector database search, network search, and user attachment content search;

[0037] Step 7: Retrieve the content of the document related to the problem from the problem-related database;

[0038] Step 8: Expand the query by adding synonyms, related concepts, or hyponyms to enrich the query content.

[0039] Step 9: Semantically reorder the search results, merge the user questions and search results, and filter out redundant information.

[0040] Step 10: Develop a real-time feedback system that allows users to provide immediate feedback during the query processing process. The system dynamically adjusts the query processing strategy based on the feedback to improve the user experience.

[0041] In some embodiments, the reasoning module and the generation module further include the following steps:

[0042] Step 1: Based on the key information extracted by the query processing module, relevant policy documents are efficiently retrieved from the vector knowledge base;

[0043] Step 2: Use a large language model to conduct in-depth semantic understanding and analysis of the retrieved documents to extract core information;

[0044] Step 3: Generate accurate and detailed answers and provide them directly to users to meet their query needs;

[0045] Step 4: Apply natural language generation technology to convert the analysis results into fluent, easy-to-understand natural language text to improve user experience;

[0046] Step 5: Check the completeness and consistency of the answer to ensure that the answer fully covers the user's query request and is consistent with the internal logic;

[0047] Step 6: Implement answer optimization, using advanced large language models to polish and adjust the generated answers to improve answer quality;

[0048] Step 7: Generate source citations for the answers, provide document links or reference information, and increase the credibility and transparency of the answers;

[0049] Step 8: Implement multi-round dialogue management. Based on user feedback and query requirements, the generated answers can guide users to the next step of the query or provide more relevant information.

[0050] Step 9: Design a strategy for generating diverse answers, providing users with multiple possible answer options to meet the needs and preferences of different users;

[0051] Step 10: Personalize answers by adjusting the style and content of answers based on user preferences and historical interaction records to improve user satisfaction.

[0052] Step 11: Conduct security and compliance checks on the answers to ensure that the information provided complies with legal and regulatory requirements and protects user privacy;

[0053] Step 12: Generate interactive answers and provide interactive elements such as links or buttons for users to further inquire or perform related operations;

[0054] Step 13: Design a feedback learning mechanism for answers. Based on user feedback and interaction data, continuously optimize and adjust the answer generation process to improve the system's intelligence level and user experience.

[0055] In some embodiments, a network retrieval module is also included.

[0056] In some embodiments, the network search module further comprises the following steps:

[0057] Step 1: Call the network search engine to conduct a broad and in-depth search based on the query keywords entered by the user;

[0058] Step 2: Carefully screen the search results, giving priority to official or authoritative information sources closely related to university policies, and extract key and valuable information from them;

[0059] Step 3: Use the extracted key information for further vector knowledge base retrieval, or directly generate answers based on this information to meet the user's query needs;

[0060] Step 4: Implement an efficient search result sorting algorithm that takes into account factors such as relevance, authority, and timeliness to ensure that users prioritize the results that best meet their needs.

[0061] Step 5: Design and apply a feedback learning mechanism to continuously adjust and optimize the search result sorting algorithm based on user click behavior and feedback data to improve user experience;

[0062] Step 6: Develop an intelligent identification system for malicious websites and false information to effectively filter out unreliable information sources and ensure that the information provided to users is authentic and trustworthy;

[0063] Step 7: Build an efficient caching system to store search results for popular queries, significantly improving system response speed and overall performance.

[0064] Step 8: Implement a multi-source data fusion mechanism to integrate results from different search engines to provide users with more comprehensive and rich answers;

[0065] Step 9: Design and implement a user privacy protection mechanism to ensure that users' personal information and privacy are fully protected during the search process;

[0066] Step 10. Develop a search result summary generator to generate a concise and clear summary for each search result to help users quickly understand the content summary.

[0067] In some implementations, a logging module is also included to monitor key time nodes and operations in the retrieval and generation process to facilitate problem location and performance optimization.

[0068] In a second aspect, the present invention proposes a method for constructing a large model for university policy question answering combined with retrieval-enhanced generation technology, comprising the following steps:

[0069] Input user questions and question-answer text vector data about university policies;

[0070] Extract keywords based on the user question language model, retrieve question-answer text vector data using a multi-level index based on the keywords, and locate policy content related to the user question based on the retrieval results;

[0071] Calculating the semantic relevance score of each document of the policy content, semantically sorting the search results according to the score, achieving a secondary evaluation of the policy content, and giving priority to providing policy content with high semantic relevance;

[0072] The policy content reasoning is performed using a combination of large and small models, including using a small model to rewrite the user question and extract keywords, wherein the question rewriting is to rewrite the user question according to the contextual conversation history and supplement the contextual information; the large model is used to summarize the detection content and user questions obtained by the positioning module and the evaluation module for reasoning to optimize computing resources and reduce latency.

[0073] Generate answers to user questions based on content reasoning and output accurate answers about university policies.

[0074] Compared with the prior art, the present invention can achieve the following beneficial technical effects:

[0075] 1) A customized dataset specifically for university policies is constructed, covering multiple aspects such as education policy, scientific research policy, and student affairs. It is stored in a vector database, and vectorization technology is used for efficient similarity search and vector clustering. Multi-level indexing technology is used to improve retrieval efficiency.

[0076] 2) Rerank technology is introduced to semantically sort search results, prioritizing semantically relevant content to enhance answer accuracy. The system also uses a combination of large and small models for inference: the small model is used for question rewriting and keyword extraction, and the large model is used for content generation, optimizing computing resources and reducing latency.

[0077] 3) The system features a logging function that monitors key operations, facilitating problem location and performance optimization. Through the aforementioned technical means, this invention can provide accurate and efficient policy information retrieval services for university teachers and students, meeting their needs for policy information in their studies and research. This aims to improve the accuracy and efficiency of university policy information retrieval.

[0078] 4) Update policy information in real time through the network search function to ensure the timeliness of information. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 Schematic diagram of the prior art search enhancement technology used in the present invention;

[0080] Figure 2 This is a module diagram of a large model for university policy question answering combined with search enhancement generation technology according to embodiment 1 of the present invention;

[0081] Figure 3 This is a flow chart of a method for constructing a large model for university policy question and answer combined with search enhancement generation technology according to embodiment 2 of the present invention;

[0082] Figure 4 This is the technical roadmap of the present invention;

[0083] Figure 5 This is a flow chart of the vector database construction method of the present invention;

[0084] Figure 6 Flowchart of the network retrieval process of the present invention. DETAILED DESCRIPTION

[0085] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0086] Figure 2The present invention shows a large-scale model for university policy question answering that combines retrieval-enhanced generation technology, including an input module, a positioning module, an evaluation module, a reasoning module, and an answer generation module.

[0087] The input module inputs a large language model of user questions about university policies and question-answer text vector data;

[0088] The positioning module extracts keywords based on the user question language model, uses a multi-level index to search the question-answer text vector data based on the keywords, and locates the policy content related to the user question based on the search results;

[0089] The evaluation module calculates the semantic relevance score of each document of the policy content, performs secondary semantic sorting on the search results based on the score, and gives priority to providing content with high semantic relevance;

[0090] The inference module uses a combination of large and small models to perform content inference. This includes using a small-parameter language model for question rewriting and keyword extraction, and a large language model for content generation, to optimize computing resources and reduce latency. The small-parameter language model can quickly and cost-effectively complete the question rewriting and keyword extraction tasks required by the positioning module. Question rewriting involves rewriting the user question based on the contextual conversation history, supplementing with default and necessary contextual information. Furthermore, the large language model aggregates relevant content obtained by the positioning and evaluation modules and the original question for inference, ensuring answer quality and accuracy.

[0091] The generation module generates answers to the user questions based on content reasoning and outputs answers about university policies.

[0092] Example 2

[0093] Figure 3 The present invention shows a method for constructing a large model of university policy question answering combined with retrieval enhancement generation technology, including the following steps:

[0094] Input a large language model of user questions about university policies and question-answer text vector data;

[0095] Extract keywords based on the user question language model, retrieve question-answer text vector data using a multi-level index based on the keywords, and locate policy content related to the user question based on the retrieval results;

[0096] Calculating the semantic relevance score of each document of the policy content, performing secondary semantic sorting on the search results based on the score, and providing content with high semantic relevance first;

[0097] Using a combination of large and small models to perform content reasoning, including using a small model for question rewriting and keyword extraction, and using a large model for content generation, to optimize computing resources and reduce latency;

[0098] Generate answers to the user questions based on content reasoning and output answers about university policies.

[0099] Figure 4 The present invention shows a technical route of a large-scale university policy question-answering model combined with retrieval enhancement generation technology, which specifically includes the following steps:

[0100] Step 1: The user first submits a question to the pre-built policy question-answering model for this university. The model takes the customized dataset as input and outputs the policy question-answering text of this university.

[0101] Step 2: The system receives the natural language text of the question submitted by the user and performs preliminary language processing, including word segmentation, stop word removal, question rewriting, etc.

[0102] Step 3: Based on the processed questions, the system uses multi-level indexing technology to quickly retrieve policy documents related to the questions. In this step, the system uses multiple indexing methods such as keyword indexing and semantic indexing to ensure the comprehensiveness and accuracy of the search results;

[0103] Step 4: The system reranks the retrieved policy documents related to the problem based on their semantic relevance. Specifically, it builds a semantic analysis engine based on the Transformer architecture, calculates the semantic relevance score of each document using cosine similarity, and sorts the policy documents related to the problem based on the score. The semantic analysis engine then performs a secondary evaluation of the documents, prioritizing the recommendation of policy documents with high semantic relevance.

[0104] Step 5: The system inputs the recommended policy documents related to the question into the large language model to generate accurate and detailed policy answers. In this step, the system comprehensively considers the content, context, and semantic relevance of the documents to generate the policy answer that best meets the user's query intent.

[0105] Step 6: The system presents the generated answer to the user and records the user's feedback. The user's feedback will be used for iterative learning and optimization of the large prediction model, specifically including: iteratively adding user feedback data to the customized dataset as input to the university policy question-answering large model;

[0106] Step 7: The system uses the logging function to record the key time nodes and operations in the entire retrieval and generation process to facilitate problem location and performance optimization.

[0107] Through the above steps, the university policy question-answering model of the present invention can provide users with an efficient and accurate policy information retrieval service, meeting the needs of university teachers and students for policy information in their learning and research.

[0108] Specifically, for the question-answer text vector knowledge base, the present invention constructs a customized data set specifically for university policies, which covers all aspects of a certain university policy, including but not limited to education policy, scientific research policy, student affairs, etc. The construction of this data set is based on an in-depth analysis and understanding of university policy information, ensuring the comprehensiveness and pertinence of the data set. The data set construction process includes collecting, screening, organizing and annotating policy documents from multiple sources, such as official announcements, regulations, instruction manuals, etc., to form a structured database. In addition, the data set also undergoes a strict quality control process to ensure the accuracy and reliability of the information. After the above-mentioned customized data set is vectorized, it serves as a question-answer text vector knowledge base.

[0109] To improve the generalization and coverage of the model, the present invention segments, expands, rewrites, extracts keywords, and summarizes the dataset. This process first involves segmenting the dataset into training, validation, and test sets to ensure that the model can be fully trained and evaluated at different stages. The dataset is expanded by adding more policy documents in relevant fields, including not only education and scientific research policies, but also other areas related to university policies such as finance, law, and international exchanges, thereby ensuring that the model can handle a wider range of queries.

[0110] While expanding the dataset, this paper prioritizes quality control. Through rewriting and proofreading, we ensure that each document conforms to standard formats and contains accurate information. Furthermore, keyword extraction from documents not only facilitates rapid indexing and retrieval but also enhances the model's understanding of the document's core content. Keyword extraction utilizes advanced text analysis techniques to automatically identify frequently occurring and informative terms within documents.

[0111] Summarization is another key step in data preprocessing. It extracts key information from documents and creates concise and clear summaries. These summaries not only help users quickly understand the main content of the document but also provide the model with rich contextual information, enabling it to more accurately locate and reference relevant information when generating answers. Furthermore, this paper employs natural language generation technology to automatically generate high-quality summaries from documents, enhancing the model's understanding and generation capabilities.

[0112] Through these comprehensive measures, the present invention ensures the richness, accuracy, and timeliness of the dataset, thereby significantly improving the performance and reliability of the model in processing university policy information.

[0113] Figure 5 The process of constructing the question-answer text vector knowledge base is shown. In a specific embodiment, the construction of the question-answer text vector knowledge base in the university policy question-answering model includes the following steps:

[0114] Step 1: Build a structured database to collect and organize documents and information related to university policies;

[0115] Step 2: Use natural language processing technology to perform semantic analysis on the document, extract key information and concepts, and convert them into vector representations and store them in the vector knowledge base;

[0116] Step 3: Segment the document content and use the large model to rewrite, expand, extract keywords, and summarize the segmented content based on the document context.

[0117] Step 4: Build a multi-level index to speed up vector database retrieval efficiency and improve retrieval quality.

[0118] Step 5: Regularly and automatically update and maintain the vector knowledge base to ensure the timeliness and accuracy of information;

[0119] Step 6: Implement an efficient retrieval algorithm to support fast query and retrieval of information in the vector knowledge base;

[0120] Step 7: Develop a user-friendly query interface that allows users to submit query requests in natural language;

[0121] Step 8. Implement data security and privacy protection mechanisms to ensure that stored information complies with legal and regulatory requirements;

[0122] Step 9: Build a multi-language support module to process query requests in different languages;

[0123] Step 10: Develop an intelligent recommendation system to recommend relevant policy information based on user historical queries and preferences;

[0124] Specifically, the use of multi-level indexing to search the question-answer text vector knowledge base further includes: the present invention uses multi-level indexing technology to store and retrieve data sets, which can significantly improve retrieval efficiency. By constructing a multi-level index structure, relevant policy information can be quickly located, thereby improving the quality and speed of document retrieval. A multi-level index is set for the vector question-answer text vector data. In the present invention, the multi-level indexing technology not only includes traditional keyword indexing, but also combines semantic indexing and ontology indexing to achieve deeper content understanding and information retrieval. A multi-level index is set for the vector question-answer text vector data, including at least a keyword index, a semantic index and an ontology index.

[0125] First, keyword indexing is built based on frequently occurring words in documents, allowing users to quickly retrieve relevant documents by entering keywords. Second, semantic indexing utilizes natural language processing technology to analyze the semantic content of documents and construct a semantic index, making search results more consistent with the user's query intent.

[0126] To further improve retrieval efficiency, the present invention also introduces an inverted index mechanism, which associates each word in a document with the document location where it appears. This allows users to quickly locate all documents containing that word when they query. Furthermore, by constructing an inverted index, the document content is inversely associated with the index terms, further improving retrieval speed.

[0127] Figure 6 The multi-level index construction process is shown. In a specific embodiment, the multi-level index construction process includes the following steps:

[0128] Step 1: Use a large language model to extract question keywords and calculate corresponding cluster labels to classify and organize questions;

[0129] Step 2: Closely integrate clustering labels with university document content to build a multi-level, structured index system to improve information retrieval efficiency;

[0130] Step 3: Build a vector database and use advanced vector search technology to achieve efficient similarity search and information retrieval functions;

[0131] Step 4: Implement document preprocessing steps, including cleaning, word segmentation, and stop word removal, to improve the quality and accuracy of the index;

[0132] Step 5: Develop a semantic understanding module, using natural language processing technology to enhance the semantic relevance of the index and improve the accuracy of search results;

[0133] Step 6: Implement a dynamic index update mechanism to ensure that the index can reflect updates and changes to policy documents in real time and maintain the timeliness of information;

[0134] Step 7: Design an index optimization algorithm and regularly evaluate and optimize the index structure to ensure the efficiency and accuracy of the index;

[0135] Step 8: Build a comprehensive security mechanism to protect sensitive information from being leaked and ensure the security of the indexing system;

[0136] Step 9: Design an extensible architecture to facilitate the subsequent addition of new index levels or types to improve the flexibility and scalability of the system;

[0137] Step 10: Develop a user-defined index function to allow users to create personalized indexes based on their own needs, thus improving the user experience.

[0138] Step 11: Build a backup and recovery system to ensure that index data can be restored in time in the event of an accident to prevent data loss or damage;

[0139] Step 12: Design a strict access control mechanism to ensure that only authorized users can access sensitive or restricted index information to ensure information security;

[0140] Step 13: Develop a semantic similarity evaluation tool to evaluate the semantic relationship between different documents and provide users with richer retrieval results and relevance analysis.

[0141] Specifically, semantically ranking the search results based on the scores to achieve secondary evaluation of the policy content further includes: To address the problem of poor semantic relevance in vector database searches, the present invention employs Rerank technology to further improve search quality. Rerank technology semantically ranks search results, prioritizing search results with high semantic relevance, thereby improving answer accuracy. The application of this technology significantly enhances the model's performance in handling complex queries, especially for polysemous and fuzzy queries.

[0142] At the heart of Rerank technology lies a small, advanced semantic analysis model that performs a secondary evaluation of the search results, specifically the policy content. This model considers not only the vector similarity between the document and the query, but also the document's content, context, and semantic relevance. This allows Rerank to identify documents that best match the user's query intent, even if they were not ranked highly in the initial vector similarity search.

[0143] During the rerank process, the model analyzes the semantic features of each search result and calculates a semantic relevance score. Documents with high scores are recommended to users first. With the introduction of rerank technology, users can find the information they need more quickly, and the model can continuously improve its retrieval and generation quality through continuous learning and optimization.

[0144] In a specific embodiment, the query processing of the university policy question-answering large model is realized through positioning and evaluation, including the following steps:

[0145] Step 1: Receive a query request input by a user;

[0146] Step 2: Process the user input to extract query intent and key information;

[0147] Step 3: Analyze the context of the query request to more accurately understand the user's intent;

[0148] Step 4: Rewrite the question based on the extracted query intent and key information;

[0149] Step 5: Convert the user’s natural language query into a machine-understandable vector representation;

[0150] Step 6: Implement query intent classification to categorize queries into predefined categories, such as vector database search, network search, and user attachment content search;

[0151] Step 7: Retrieve the content of the document related to the problem from the problem-related database;

[0152] Step 8: Expand the query by adding synonyms, related concepts, or hyponyms to enrich the query content.

[0153] Step 9: Semantically reorder the search results, merge the user questions and search results, and filter out redundant information.

[0154] Step 10: Develop a real-time feedback system that allows users to provide immediate feedback during the query processing process. The system dynamically adjusts the query processing strategy based on the feedback to improve the user experience.

[0155] Specifically, the strategic content reasoning is performed in a manner combining large and small models, further comprising: in order to improve the reasoning speed, the present invention uses a combination of large and small models for reasoning. This strategy aims to optimize the use of computing resources while ensuring the efficiency of the reasoning process and the quality of generated content. In the early stages of processing user queries, such as question rewriting and keyword extraction, the present invention uses lightweight small models for reasoning. These small models are specially designed to quickly process text data and reduce the computational burden, thereby significantly reducing the end-to-end delay from user input to preliminary processing results. Among them, the large and small models refer to the large language model and the small language model, respectively.

[0156] Small models offer fast inference speeds and are suitable for tasks requiring high real-time performance, such as quickly identifying keywords and phrases in queries and performing preliminary rewriting and reframing of queries to enhance their searchability. Furthermore, small models can effectively perform information filtering and preliminary classification, laying the foundation for subsequent in-depth processing.

[0157] During the content generation phase, the present invention switches to large-scale models for inference. These large-scale models have more parameters and more complex network structures, enabling in-depth semantic understanding and high-quality text generation. Building on the work of the earlier small-scale models, large-scale models leverage their powerful language generation capabilities to produce accurate, detailed, and contextually relevant responses.

[0158] To achieve seamless collaboration between large and small models, the present invention dynamically selects the most appropriate model for inference based on task requirements, ensuring both fast and accurate inference. Furthermore, the present invention introduces a model caching strategy to cache frequently used small models to further improve inference speed.

[0159] In a specific embodiment, the university policy question answering big model infers and generates answers to the university policy question answering big model, including the following steps:

[0160] Step 1: Based on the key information extracted by the query processing module, relevant policy documents are efficiently retrieved from the vector knowledge base;

[0161] Step 2: Use a large language model to conduct in-depth semantic understanding and analysis of the retrieved documents to extract core information;

[0162] Step 3: Generate accurate and detailed answers and provide them directly to users to meet their query needs;

[0163] Step 4: Apply natural language generation technology to convert the analysis results into fluent, easy-to-understand natural language text to improve user experience;

[0164] Step 5: Check the completeness and consistency of the answer to ensure that the answer fully covers the user's query request and is consistent with the internal logic;

[0165] Step 6: Implement answer optimization, using advanced large language models to polish and adjust the generated answers to improve answer quality;

[0166] Step 7: Generate source citations for the answers, provide document links or reference information, and increase the credibility and transparency of the answers;

[0167] Step 8: Implement multi-round dialogue management. Based on user feedback and query requirements, the generated answers can guide users to the next step of the query or provide more relevant information.

[0168] Step 9: Design a strategy for generating diverse answers, providing users with multiple possible answer options to meet the needs and preferences of different users;

[0169] Step 10: Personalize answers by adjusting the style and content of answers based on user preferences and historical interaction records to improve user satisfaction.

[0170] Step 11: Conduct security and compliance checks on the answers to ensure that the information provided complies with legal and regulatory requirements and protects user privacy;

[0171] Step 12: Generate interactive answers and provide interactive elements such as links or buttons for users to further inquire or perform related operations;

[0172] Step 13: Design a feedback learning mechanism for answers. Based on user feedback and interaction data, continuously optimize and adjust the answer generation process to improve the system's intelligence level and user experience.

[0173] In summary, by using a combination of large and small models for reasoning, the present invention ensures the accuracy and effectiveness of the generated content while ensuring the reasoning speed, providing users with an efficient and reliable university policy information retrieval service.

[0174] Furthermore, to address the issue of data outdatedness in large language models, the present invention adds a web search function. This function allows the model to access the latest university policy information in real time, ensuring that the answers provided to users are based on the latest policies. This function relies on an efficient web crawler system that regularly accesses university official websites and other trusted policy publishing platforms to capture the latest policy documents and announcements.

[0175] At the heart of the web search function lies an intelligent crawler scheduling system, which automatically determines the frequency and scope of crawling based on pre-set rules and strategies. To ensure data accuracy and authority, the system only crawls verified official university resources. Furthermore, the system features automatic verification and deduplication capabilities, ensuring that stored datasets contain no outdated or duplicate information.

[0176] Furthermore, to further enhance the practicality of web search functionality, this invention integrates text parsing and data cleaning modules. These modules extract structured policy information from crawled web pages and convert it into a unified format for better understanding and utilization by the model. Furthermore, the data cleaning module removes irrelevant content and noise, ensuring the quality and usability of the dataset.

[0177] Furthermore, the present invention incorporates a real-time update mechanism. When new policy information is retrieved from the network, the system automatically updates the knowledge base and triggers an index update process, ensuring that users have immediate access to the latest policy content. This mechanism not only improves the system's responsiveness to policy changes but also enhances the timeliness and accuracy of the model's query processing.

[0178] In summary, by adding the network retrieval function, the present invention not only solves the problem of outdated large language model data, but also provides users with a university policy information retrieval tool that is always based on the latest policy information through a real-time update mechanism and an efficient data processing flow.

[0179] Furthermore, the present invention includes a logging function. The system built with this function monitors key time points and operations during the retrieval and generation process, facilitating problem location and performance optimization. Logging is a crucial component of system stability and reliability, providing valuable data support for system operation and maintenance.

[0180] This logging feature utilizes advanced data collection technology to capture various events and operations during system operation in real time. This log information includes, but is not limited to, user queries, search requests, data processing, model inference results, and system response time. By recording this detailed data, system administrators can gain a comprehensive understanding of the system's operational status and promptly identify and resolve potential issues.

[0181] To improve the readability and usability of log data, this paper uses structured log formats, such as JSON, to facilitate log storage, querying, and analysis. Furthermore, the system provides a log analysis tool that automatically analyzes log data, identifies common problem patterns, such as performance bottlenecks and abnormal requests, and generates corresponding reports.

[0182] The logging feature also supports custom configuration, allowing administrators to select the level of detail and log retention period as needed. This not only helps protect user privacy but also helps optimize storage resource usage. The system also features real-time monitoring and alerting mechanisms. When it detects that the execution time of critical operations exceeds preset thresholds or anomalies occur, the system automatically triggers an alarm, notifying operations and maintenance personnel to address the situation.

[0183] Through the logging function, the present invention not only improves the maintainability and scalability of the system, but also provides users with a more stable and reliable service experience. The detailed recording and analysis of log data provides a solid data foundation for the continuous improvement and optimization of the system.

[0184] To sum up, through the above-mentioned technical means, the university policy question-answering model based on retrieval enhancement generation of the present invention can provide more accurate and efficient policy information retrieval services, and meet the needs of university teachers and students for policy information in their learning and research.

[0185] The present invention's large-scale model and construction method for university policy question-answering, combined with retrieval-enhanced generation technology, primarily addresses the low accuracy and inefficiency of existing large-scale language models in the field of university policy question-answering. It proposes a highly efficient and accurate method for retrieving university policy information. This method leverages advanced retrieval-enhanced technology, combined with customized datasets and a real-time update mechanism, to provide more accurate and efficient policy information retrieval services.

[0186] Example

[0187] Example 2

[0188] Figure 3 A novel vector database construction method is shown. Based on Example 1, Example 2 of the present invention also provides a novel vector database construction method. Targeting university policy documents, the present invention employs a strategy of segmentation, expansion, keyword extraction, abstract summarization, and multi-level indexing to construct a database, thereby improving the retrieval efficiency and quality of the vector database. The specific steps are:

[0189] Step 1: Segment the university policy documents into multiple segments, each of which contains sufficiently rich semantic information (for example, each segment does not exceed 1,000 words) to facilitate subsequent processing and index construction.

[0190] Step 2: Expand the segmented document segments by adding policy documents in related fields, such as finance, law, and international exchanges, to enhance the generalization ability and coverage of the model.

[0191] Step 3: Extract keywords from each document fragment and use the TF-IDF algorithm to automatically identify frequently appearing and informative words in the document for fast indexing and retrieval, while also enhancing the model's grasp of the document's core content.

[0192] Step 4: Summarize the document fragments and extract key information to form concise and clear summaries. These summaries help users quickly understand the main content of the document and also provide rich contextual information for the big prediction model, so that it can more accurately locate and quote relevant information when generating answers.

[0193] Step 5: Build a multi-level index, including traditional keyword indexing, semantic indexing (based on the BERT pre-trained model), and ontology indexing (based on the policy domain ontology) to achieve deeper content understanding and information retrieval. Multi-level indexing technology can significantly improve retrieval efficiency. By building a multi-level index structure, relevant policy information can be quickly located, thereby improving the quality and speed of document retrieval.

[0194] Step 6: To further improve retrieval efficiency, the present invention also introduces a reverse index mechanism, which associates each word in the document with the document location where it appears, so that all documents containing the word can be quickly located when a user queries.

[0195] Step 7: Build an inverted index to reversely associate document content with index items, further improving retrieval speed and accuracy.

[0196] Through the above steps, the present invention ensures the richness, accuracy and timeliness of the vector database, thereby significantly improving the performance and reliability of the model in processing university policy information.

[0197] Example 3

[0198] Based on Example 1 and Example 2, Example 3 of the present invention provides an optimization of network retrieval, such as Figure 4 As shown, the search is mainly based on search engines, and the URLs of the search results are filtered to ensure that the source is an official website. Then the web page content is extracted and summarized. Finally, the large language model determines whether it is relevant to the question. If so, the search results are returned. The specific steps are as follows:

[0199] Step 1: The user submits a query request to the system, and the system records the user's query keywords and timestamp.

[0200] Step 2: The system calls the search engine API (such as Baidu API), searches using the user's query keywords, and obtains an initial search result list.

[0201] Step 3: The system filters the initial search results by URL, using pre-set rules and strategies to select search results from official websites. In this step, the system checks the URL, domain name, page content, and other information to ensure it is an official or trustworthy source.

[0202] Step 4: The system extracts the content of the screened official web pages using web content extraction tools (such as the Beautiful Soup library) to extract the core content and key information from the web pages.

[0203] Step 5: The system calls the large language model API and submits the extracted web page content and the user's original query to the large language model, which performs semantic analysis and relevance judgment.

[0204] Step 6: The large language model analyzes the submitted content to determine whether it is relevant to the user's query. If so, the system returns the relevant content to the user; if not, the system returns a prompt to inform the user that no relevant content was found and recommends that the user try another query or use different keywords.

[0205] Step 7: The system records user feedback and the relevance score of the search results for iterative learning and optimization of the model.

[0206] Through the above steps, the university policy question-answering model of the present invention can provide more accurate and efficient policy information retrieval services, meeting the policy information needs of university teachers and students in their learning and research.

[0207] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.

[0208] It should be noted that although the present invention has been shown and described with reference to specific exemplary embodiments of the present invention, those skilled in the art should understand that the present invention is not limited to the above-mentioned embodiments and all kinds of changes to the present invention fall within the scope of protection of the present invention.

Claims

1. A large-scale model for university policy question answering that combines retrieval-enhanced generation technology, characterized by: It includes input module, positioning module, evaluation module, reasoning module and question-answer generation module, among which: The input module inputs user questions and question-answer text vector data about university policies; The positioning module extracts keywords based on the user question language model, uses a multi-level index to search the question-answer text vector data based on the keywords, and locates the policy content related to the user question based on the search results; The evaluation module calculates the semantic relevance score of each document of the policy content, and semantically sorts the search results according to the score to achieve a secondary evaluation of the policy content, giving priority to providing policy content with high semantic relevance; The reasoning module uses a combination of large and small models to perform policy content reasoning, including using the small model to rewrite the user question and extract keywords, where the question rewriting is to rewrite the user question based on the contextual conversation history and supplement the contextual information; and uses the large model to summarize the detected content and user questions obtained by the positioning module and the evaluation module for reasoning, thereby optimizing computing resources and reducing latency. The generation module generates answers to the user questions based on content reasoning and outputs accurate answers about university policies.

2. The large-scale university policy question-answering model combined with search-enhanced generation technology according to claim 1 is characterized in that: The vector knowledge base includes a constructed customized data set of university policies, which at least covers question-answer text vector data including education policy, scientific research policy and student affairs, and sets a multi-level index for the vector question-answer text vector data, including at least keyword index, semantic index and ontology index.

3. The large-scale university policy question-answering model combined with search-enhanced generation technology according to claim 1 is characterized in that: The multi-level index construction further comprises the following steps: Step 1: Use a large language model to extract question keywords and calculate corresponding cluster labels to classify and organize questions; Step 2: Closely integrate clustering labels with university document content to build a multi-level, structured index system to improve information retrieval efficiency; Step 3: Build a vector database and use advanced vector search to achieve efficient similarity search and information retrieval functions; Step 4: Implement document preprocessing steps, including at least cleaning, word segmentation, and stop word removal, to improve the quality and accuracy of the index; Step 5: Develop a semantic understanding module, using natural language processing technology to enhance the semantic relevance of the index and improve the accuracy of search results; Step 6: Implement a dynamic index update mechanism to ensure that the index can reflect updates and changes to policy documents in real time and maintain the timeliness of information; Step 7: Design an index optimization algorithm and regularly evaluate and optimize the index structure to ensure the efficiency and accuracy of the index; Step 8: Build a comprehensive security mechanism to protect sensitive information from being leaked and ensure the security of the indexing system; Step 9: Design an extensible architecture to facilitate the subsequent addition of new index levels or types to improve the flexibility and scalability of the system; Step 10: Develop a user-defined index function to allow users to create personalized indexes based on their own needs, thus improving the user experience. Step 11: Build a backup and recovery system to ensure that index data can be restored in time in the event of an accident to prevent data loss or damage; Step 12: Design a strict access control mechanism to ensure that only authorized users can access sensitive or restricted index information to ensure information security; Step 13: Develop a semantic similarity evaluation tool to evaluate the semantic relationship between different documents and provide users with richer retrieval results and relevance analysis.

4. The large-scale university policy question-answering model combined with search-enhanced generation technology according to claim 1 is characterized in that: The positioning module and the evaluation module further include the following steps: Step 1: Receive a query request input by a user; Step 2: Process the user input to extract query intent and key information; Step 3: Analyze the context of the query request to more accurately understand the user's intent; Step 4: Rewrite the question based on the extracted query intent and key information; Step 5: Convert the user’s natural language query into a machine-understandable vector representation; Step 6: Implement query intent classification to classify queries into predefined categories, including vector database search, network search, and user attachment content search; Step 7: Retrieve the content of the document related to the problem from the problem-related database; Step 8: Expand the query by adding synonyms, related concepts, or hyponyms to enrich the query content. Step 9: Semantically reorder the search results, merge the user questions and search results, and filter out redundant information. Step 10: Develop a real-time feedback system that allows users to provide immediate feedback during the query processing process. The system dynamically adjusts the query processing strategy based on the feedback to improve the user experience.

5. The large-scale university policy question-answering model combined with search-enhanced generation technology according to claim 1 is characterized in that: The reasoning module and the generation module further include the following steps: Step 1: Based on the key information extracted by the query processing module, relevant policy documents are efficiently retrieved from the vector knowledge base; Step 2: Use a large language model to conduct in-depth semantic understanding and analysis of the retrieved documents to extract core information; Step 3: Generate accurate and detailed answers and provide them directly to users to meet their query needs; Step 4: Apply natural language generation technology to convert the analysis results into fluent, easy-to-understand natural language text to improve user experience; Step 5: Check the completeness and consistency of the answers to ensure that they fully cover the user's query and are consistent in their internal logic. Step 6: Implement answer optimization, using advanced large language models to polish and adjust the generated answers to improve answer quality; Step 7: Generate source citations for the answers, provide document links or reference information, and increase the credibility and transparency of the answers; Step 8: Implement multi-round dialogue management. Based on user feedback and query requirements, the generated answers can guide users to the next step of the query or provide more relevant information. Step 9: Design a strategy for generating diverse answers, providing users with multiple possible answer options to meet the needs and preferences of different users; Step 10: Personalize answers by adjusting the style and content of answers based on user preferences and historical interaction records to improve user satisfaction. Step 11: Conduct security and compliance checks on the answers to ensure that the information provided complies with legal and regulatory requirements and protects user privacy; Step 12: Generate interactive answers and provide links or buttons for users to conduct further inquiries or perform related operations; Step 13: Design a feedback learning mechanism for answers. Based on user feedback and interaction data, continuously optimize and adjust the answer generation process to improve the system's intelligence level and user experience.

6. The university policy question-answering model combined with search-enhanced generation technology according to claim 1 is characterized in that: Also includes a network search module.

7. The large-scale university policy question-answering model combined with search-enhanced generation technology according to claim 1 is characterized in that: The network retrieval module further comprises the following steps: Step 1: Call the network search engine to conduct a broad and in-depth search based on the query keywords entered by the user; Step 2: Carefully screen the search results, giving priority to official or authoritative information sources closely related to university policies, and extract key and valuable information from them; Step 3: Use the extracted key information for further vector knowledge base retrieval, or directly generate answers based on this information to meet the user's query needs; Step 4: Implement an efficient sorting algorithm for search results, taking into account relevance, authority, and timeliness to ensure that users see the results that best meet their needs first. Step 5: Design and apply a feedback learning mechanism to continuously adjust and optimize the search result sorting algorithm based on user click behavior and feedback data to improve user experience; Step 6: Develop an intelligent identification system for malicious websites and false information to effectively filter out unreliable information sources and ensure that the information provided to users is authentic and trustworthy; Step 7: Build an efficient caching system to store search results for popular queries, significantly improving system response speed and overall performance. Step 8: Implement a multi-source data fusion mechanism to integrate results from different search engines to provide users with more comprehensive and rich answers; Step 9: Design and implement a user privacy protection mechanism to ensure that users' personal information and privacy are fully protected during the search process; Step 10. Develop a search result summary generator to generate a concise and clear summary for each search result to help users quickly understand the content summary.

8. The university policy question-answering model combined with search-enhanced generation technology according to claim 1 is characterized in that: It also includes a logging module to monitor key time nodes and operations in the retrieval and generation process to facilitate problem location and performance optimization.

9. The method for constructing a large university policy question-answering model combined with search-enhanced generation technology according to any one of claims 1 to 8, characterized in that: The method comprises the following steps: Input user questions and question-answer text vector data about university policies; Extract keywords based on the user question language model, retrieve question-answer text vector data using a multi-level index based on the keywords, and locate policy content related to the user question based on the retrieval results; Calculating the semantic relevance score of each document of the policy content, semantically sorting the search results according to the score, achieving a secondary evaluation of the policy content, and giving priority to providing policy content with high semantic relevance; The reasoning module uses a combination of large and small models to perform policy content reasoning, including using the small model to rewrite the user question and extract keywords, where the question rewriting is to rewrite the user question based on the contextual conversation history and supplement the contextual information; and uses the large model to summarize the detected content and user questions obtained by the positioning module and the evaluation module for reasoning, thereby optimizing computing resources and reducing latency. Generate answers to user questions based on content reasoning and output accurate answers about university policies.

Images