Vertical field intelligent question answering method and system based on hierarchical labeling and screening ranking

Abstract

This invention belongs to the field of intelligent question answering technology, and discloses a vertical domain intelligent question answering method and system based on hierarchical annotation and filtering ranking. The method includes: acquiring and preprocessing knowledge text data in the vertical domain; constructing a vector database and index based on the preprocessed knowledge text data; rewriting the question text data using a large language model; obtaining several candidate texts from the vector database based on the vector matching algorithm after vectorizing the rewritten question text data; evaluating the relevance and performing hierarchical annotation on the candidate texts based on a large language model, and filtering and ranking the candidate texts according to the annotation results to generate highly reliable question answering results. This invention, by combining large model technology, retrieval enhancement generation technology, and intelligent filtering and ranking mechanisms, significantly improves the accuracy of retrieval results, thereby enhancing the reliability and interpretability of question answering results, and effectively improving the user's interactive experience.

Description

Technical Field

[0001] This invention relates to the field of intelligent question answering technology, and in particular to a vertical domain intelligent question answering method and system based on hierarchical labeling and filtering sorting. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent question-answering systems have been widely applied across various industries. Vertical domain intelligent question-answering systems are also gradually becoming an important part of information services. In vertical domains, users typically require efficient and accurate retrieval and answers when searching for relevant information. However, existing vertical domain intelligent question-answering methods and systems still have certain limitations in handling complex queries and providing highly accurate answers, resulting in low information retrieval efficiency and impacting user experience and system usability.

[0003] Chinese Patent Publication No. CN118194993A discloses a digital campus intelligent question-answering service method and system based on LLM, including: preprocessing campus data; after preprocessing, segmenting the preprocessed text data into multiple text blocks; constructing an embedding model; inputting first text vector data into the embedding model for vectorization to obtain second text vector data, which can reduce the dimensionality of the second text vector; when a user needs to query, the embedding vector corresponding to the first input question can be obtained; based on similarity calculation, multiple matching answers most similar to the embedding vector are matched from the second text vector data; and then the first input question and multiple matching answers are used as context input into the LLM model to output the answer text information.

[0004] This technical solution primarily uses vector results from preprocessed campus data to match multiple answers based on similarity, and then inputs these matches into an LLM model to generate the final answer. However, after obtaining multiple matching answers based on similarity calculation, it does not effectively filter these matching answers but directly inputs them into the LLM model to generate the final answer. Consequently, since the retrieved matching answers may contain a large amount of irrelevant or low-relevance information, this leads to a decrease in the accuracy of the answers generated by the large language model, affecting the reliability of the question-answering system. Summary of the Invention

[0005] This invention provides a vertical domain intelligent question answering method and system based on hierarchical labeling and filtering sorting to solve the problems of the above-mentioned technologies in the prior art.

[0006] The purpose of this invention is to address the problems in the prior art by providing a vertical domain intelligent question-answering method and system based on hierarchical labeling and filtering sorting, which significantly improves the accuracy of search results, thereby enhancing the reliability and interpretability of question-answering results and effectively improving the user interaction experience.

[0007] According to a first aspect of the present invention, a vertical domain intelligent question answering method based on hierarchical annotation and filtering sorting is provided, comprising the following steps:

[0008] S1. Acquire knowledge text data in the vertical domain and preprocess it. Construct a vector database and index based on the preprocessed knowledge text data.

[0009] S2. Collect the question text data input by the user, and rewrite the question text data using a large language model to optimize the expression.

[0010] S3. Vectorized rewritten problem text data, based on vector matching algorithm, obtains several candidate texts from vector database;

[0011] S4. Based on the large language model 2, the candidate texts are relevance evaluated and hierarchically labeled. The candidate texts are then filtered and sorted according to the labeling results to generate highly credible question-and-answer results.

[0012] Furthermore, the process of acquiring and preprocessing knowledge text data in the vertical domain, and then constructing a vector database and index based on the preprocessed knowledge text data, includes the following steps:

[0013] S11. Obtain knowledge text data in the vertical field, and perform document parsing and conversion of non-text formats to plain text formats;

[0014] S12. Use a semantic blocker to perform semantic-based block processing on the format-converted knowledge text data in order to extract information units with independent semantics.

[0015] S13. Based on the text embedding model, the knowledge text block is transformed into a vector, and the transformed vector is normalized to eliminate vector scale differences.

[0016] S14. Establish a mapping relationship between knowledge text blocks and their corresponding vector representations using an index structure, and store the index structure and vectorized data in a vector database to support efficient retrieval and similarity matching.

[0017] Furthermore, the expression for the vectorization transformation is:

[0018] V i =f(T) i )

[0019] The expression for normalization is:

[0020]

[0021] In the formula, T iLet V represent the i-th knowledge text block in the input, f() represent the encoding function of the text embedding model, and V i Let represent the high-dimensional vector corresponding to the i-th knowledge text block. ||V represents the normalized vector corresponding to the i-th knowledge text block. i || represents a high-dimensional vector V i The Euclidean norm, V i，j Represents a high-dimensional vector V i The j-th component, where n represents the total number of knowledge text blocks.

[0022] Furthermore, the process of collecting user-input question text data and rewriting it using a large language model to optimize its expression includes the following steps:

[0023] S21. Receive the original question text data input by the user, and use the preset prompt word template to concatenate the prompt words and question text into a complete input sequence;

[0024] S22. Based on prompt words, the problem text is rewritten and optimized using a large language model after a pair of concatenated input sequences, generating semantically complete and compliant rewritten text.

[0025] Furthermore, the expression for the problem text data rewriting process is as follows:

[0026] Rewrite_Question=LLM(Prompt;Original_Question;θ)

[0027] In the formula, Rewrite_Question represents the rewritten user question text, LLM() represents the calculation of Big Language Model 1, Prompt represents the prompt word, Original_Question represents the original question text input by the user, and θ represents the parameters of Big Language Model 1.

[0028] Furthermore, the vectorized rewritten problem text data, based on a vector matching algorithm, obtains several candidate texts from the vector database, including the following steps:

[0029] S31. Use a text embedding model to vectorize the rewritten problem text data, generate the corresponding high-dimensional vector representation, and normalize the generated high-dimensional vector.

[0030] S32. Use a vector matching algorithm to perform similarity retrieval in the vector database, calculate the similarity between the question text vector and the text vectors stored in the vector database, and select several candidate texts with high matching degree.

[0031] Furthermore, the vector matching algorithm includes either a cosine similarity algorithm or a similarity algorithm based on Euclidean distance.

[0032] Furthermore, the process of evaluating the relevance and performing hierarchical annotation on candidate texts based on the large language model 2, and then filtering and ranking the candidate texts according to the annotation results to generate highly reliable question-and-answer results includes the following steps:

[0033] S41. Use the Large Language Model 2 to perform relevance analysis on the candidate texts, and label the candidate texts with relevance based on the degree of matching between the candidate texts and the question texts.

[0034] S42. Based on the relevance annotation results of the candidate texts, filter the candidate texts to obtain reference texts, and sort the reference texts in order of relevance from high to low.

[0035] S43. Hint: In the second language model, the reference texts are input in descending order of relevance to generate highly reliable question-and-answer results.

[0036] Furthermore, the expression for relevance evaluation and hierarchical annotation of candidate texts based on the Large Language Model 2 is as follows:

[0037] Relevance_Tag=LLM'(Prompt; Question; Candidate_Text; θ')

[0038] In the formula, Relevance_Tag represents the annotation result of the candidate text, LLM′() represents the calculation of the Large Language Model 2, Prompt represents the prompt word, Question represents the question entered by the user, Candidate_Text represents a candidate text, and θ′ represents the parameters of the Large Language Model 2.

[0039] According to a second aspect of the present invention, a vertical domain intelligent question answering system based on hierarchical annotation and filtering sorting is provided, including a vector database construction module, a question text data rewriting module, a retrieval matching module and a question answering result generation module;

[0040] The vector database construction module is used to acquire knowledge text data in the vertical domain and preprocess it, and then construct a vector database and index based on the preprocessed knowledge text data.

[0041] The question text data rewriting module is used to collect the question text data input by the user and rewrite the question text data using a large language model to optimize the expression.

[0042] The retrieval and matching module is used to obtain several candidate texts from the vector database based on the vector matching algorithm after the vectorized rewritten problem text data.

[0043] The question-and-answer result generation module is used to evaluate the relevance of candidate texts and perform hierarchical annotation based on the large language model 2, and to filter and sort the candidate texts according to the annotation results in order to generate highly credible question-and-answer results.

[0044] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0045] This invention introduces a retrieval-enhanced generation technique, enabling real-time retrieval and acquisition of the latest information from external knowledge bases. By rewriting user question text using a large language model and corresponding prompt words, the expression is optimized, improving the accuracy of retrieval results. Furthermore, a large language model is used to evaluate the relevance and hierarchically label candidate texts, and based on the labeling results, the candidate texts are filtered and ranked to prioritize highly relevant information, thereby generating highly reliable question-and-answer results. Therefore, this invention significantly enhances the reliability and interpretability of question-and-answer results, effectively improving the user's interactive experience.

[0046] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0048] Figure 1 This is a flowchart illustrating a vertical domain intelligent question answering method based on hierarchical annotation and filtering sorting, according to an exemplary embodiment.

[0049] Figure 2 This is a schematic diagram illustrating a vertical domain intelligent question answering method based on hierarchical annotation and filtering sorting, according to an exemplary embodiment.

[0050] Figure 3 This is a detailed architecture diagram of a vertical domain intelligent question answering method based on hierarchical annotation and filtering sorting, according to an exemplary embodiment.

[0051] Figure 4 This is a structural block diagram of a vertical domain intelligent question-answering system based on hierarchical labeling and filtering sorting, according to an exemplary embodiment. Detailed Implementation

[0052] The following description and accompanying drawings fully illustrate specific embodiments described herein to enable those skilled in the art to practice them. Some embodiments may include or substitute parts and features of other embodiments. The scope of the embodiments herein encompasses the entire scope of the claims and all available equivalents thereof. Throughout this document, the terms “first,” “second,” etc., are used only to distinguish one element from another without requiring or implying any actual relationship or order between the elements. Indeed, a first element can also be referred to as a second element, and vice versa. Furthermore, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a structure, apparatus, or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a structure, apparatus, or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the structure, apparatus, or device that includes said element. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0053] In this document, unless otherwise stated, the term "multiple" means two or more.

[0054] In this article, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0055] In this article, the term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0056] It should be understood that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order constraint on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the diagram may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0057] The modules in the apparatus or system of this application can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0058] The relevant technical terms in this invention are explained as follows:

[0059] Explanation of Retrieval-Augmented Generation (RAG): Retrieval-Augmented Generation (RAG) is a natural language processing technique that integrates information retrieval and text generation. RAG methods typically include two core stages: First, using retrieval techniques such as vector search or keyword matching, documents or fragments highly relevant to the user's query are retrieved from an external knowledge base; second, the retrieved content is input as contextual information into the text generation model to generate output that better meets the user's needs and provides more comprehensive information. This method enhances the accuracy and reliability of text generation by introducing external knowledge bases or document storage during the content generation process of the generative model (such as a large language model) to retrieve relevant information.

[0060] Explanation of Large Language Models (LLMs): Large Language Models (LLMs) are a class of deep learning-based Natural Language Processing (NLP) models. They typically employ large-scale neural network architectures (such as Transformers) and are pre-trained on massive amounts of text data to learn the syntax, semantics, and contextual relationships of a language. The core advantage of Large Language Models lies in their powerful contextual understanding and generalization capabilities. These models generate high-dimensional word vector representations through autoregressive or autoencoder methods, enabling them to perform various language understanding and generation tasks such as text generation, machine translation, question answering systems, and summary generation.

[0061] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0062] Figures 1-3 An embodiment of the vertical domain intelligent question answering method based on hierarchical labeling and filtering sorting of the present invention is shown.

[0063] In this optional embodiment, the vertical domain intelligent question answering method based on hierarchical annotation and filtering sorting includes the following steps:

[0064] S1. Acquire knowledge text data in the vertical domain and preprocess it. Construct a vector database and index based on the preprocessed knowledge text data.

[0065] The steps of acquiring and preprocessing knowledge text data in a vertical domain, and then constructing a vector database and index based on the preprocessed knowledge text data, include the following:

[0066] S11. Obtain knowledge text data in the vertical field, and perform document parsing and conversion of non-text format to plain text format to extract plain text content.

[0067] Specifically, the document type uploaded by the user is identified based on the document's file extension, and the corresponding loader is used for parsing and processing. If the document type is JSON, the JSONReader in the LlamaIndex framework is used for parsing, and the processed data is stored; if the document type is WORD, PDF, or plain text, the SimpleDirectoryReader loader provided by LlamaIndex is used for parsing, and the corresponding text data is stored.

[0068] S12. Use a semantic blocker to perform semantic-based block processing on the format-converted knowledge text data in order to extract information units with independent semantics.

[0069] The semantic splitter node parser provided by LlamaIndex is used to perform semantic-based chunking of text to identify and extract information units with independent semantics, thereby improving the semantic integrity of text chunks and the accuracy of retrieval matching.

[0070] LlamaIndex employs an embedding-based semantic chunking strategy. Its core mechanism is to calculate text similarity based on a sliding window and merge adjacent sentences with similarity reaching a set threshold into the same text chunk to ensure that content with strong semantic relevance is grouped into the same unit, thus guaranteeing the semantic integrity of the text chunk.

[0071] During similarity calculation, an embedding model is required for text vectorization. Suitable models include BERT, RoBERTa, or GPT. Those skilled in the art can choose the appropriate embedding model based on specific application requirements and available computing resources.

[0072] S13. Based on the text embedding model, the knowledge text block is transformed into a vector, and the transformed vector is normalized to eliminate vector scale differences.

[0073] Specifically, the text block is vectorized using a text embedding model. The process is as follows: the text block is input into the text embedding model, and the model calculates and outputs the corresponding high-dimensional vector. The specific expression is as follows:

[0074] V i =f(T)i )

[0075] In the formula, T i Let V represent the i-th knowledge text block in the input, f() represent the encoding function of the text embedding model, and V i This represents the high-dimensional vector corresponding to the i-th knowledge text block;

[0076] Specifically, a normalization operation is performed on the generated vector. Given a generated vector, the normalization calculation formula is as follows:

[0077]

[0078] In the formula, ||V represents the normalized vector corresponding to the i-th knowledge text block. i || represents a high-dimensional vector V i The Euclidean norm (L2 norm), V i，j Represents a high-dimensional vector V i The j-th component, where n represents the total number of knowledge text blocks.

[0079] S14. Establish a mapping relationship between knowledge text blocks and their corresponding vector representations using an index structure, and store the index structure and vectorized data in a vector database to support efficient retrieval and similarity matching.

[0080] Specifically, existing text embedding models are used to transform each text block into a vector, and the transformed vector data is normalized to eliminate vector scale differences, thereby improving the accuracy of vector similarity calculation. Subsequently, the text blocks and their corresponding vector data are stored in a vector database, and an index is built to support efficient retrieval and matching operations.

[0081] S2. Collect the question text data input by the user, and rewrite the question text data using a large language model to optimize the expression.

[0082] When a user asks a question, the system receives the user's input question and answer request and preprocesses the input text, including: receiving the user's original question text and preprocessing it, wherein the preprocessing includes rewriting the user's question text using a large language model and corresponding prompt words to optimize the expression and improve the accuracy of retrieval and answer generation;

[0083] The process of collecting user-input question text data and rewriting it using a large language model to optimize its expression includes the following steps:

[0084] S21. Receive the original question text data input by the user, and use the preset prompt word template to concatenate the prompt words and question text into a complete input sequence;

[0085] S22. Based on prompt words, the problem text is rewritten and optimized using a large language model after a pair of concatenated input sequences, generating semantically complete and compliant rewritten text.

[0086] Specifically, the concatenated input sequence is input into the large language model 1 for calculation. The model rewrites and optimizes the user's question text based on the prompt words, generating a semantically complete and compliant rewritten text. The rewritten text generated by the large language model 1 is extracted and provided as standardized question text to the subsequent vectorization and retrieval modules.

[0087] The expression for the problem text data rewriting process is:

[0088] Rewrite_Question=LLM(Prompt;Original_Question;θ)

[0089] In the formula, Rewrite_Question represents the rewritten user question text, LLM() represents the calculation of Big Language Model 1, Prompt represents the prompt word, Original_Question represents the original question text input by the user, and θ represents the parameters of Big Language Model 1.

[0090] The above-mentioned prompt mainly suggests that the large language model rewrite the user's question text. Below is a sample prompt:

[0091] Please rewrite the following user question text to optimize its expression and improve the accuracy of retrieval and answer generation. Ensure semantic consistency and eliminate ambiguity as much as possible.

[0092] User question text: {Original_Question};

[0093] The rewritten problem text:

[0094] The rewritten question text should be clearer, more precise, and conform to the norms of natural language expression.

[0095] This prompt is for reference only, and those skilled in the art can adjust it according to the actual situation of the Large Language Model 1.

[0096] S3. Vectorized rewritten problem text data, based on vector matching algorithm, obtains several candidate texts from vector database;

[0097] The vectorized rewritten problem text data, based on a vector matching algorithm, obtains several candidate texts from a vector database, including the following steps:

[0098] S31. Use a text embedding model (in this embodiment, the text embedding model can be BERT, RoBERTa, or GPT, but it needs to be consistent with the previous text embedding model) to vectorize the rewritten problem text data, generate the corresponding high-dimensional vector representation, and normalize the generated high-dimensional vector.

[0099] Specifically, a text embedding model is used to vectorize the preprocessed user question text, generating a corresponding high-dimensional vector representation. The specific process is as follows: the preprocessed user question text is input into the text embedding model, and the model calculates and outputs the corresponding high-dimensional vector. The specific expression is:

[0100] V = f(T)

[0101] In the formula, T represents the preprocessed user question text as input, f() represents the encoding function of the text embedding model, and V represents the high-dimensional vector corresponding to the preprocessed user question text.

[0102] Specifically, a normalization operation is performed on the generated vector. Given a generated vector V, its normalization calculation formula is:

[0103]

[0104] In the formula, V norm Let ||V|| represent the normalized vector corresponding to vector V, where V represents the high-dimensional vector corresponding to the preprocessed user question text, and ||V|| represents the Euclidean norm (L2 norm) of vector V. j This represents the j-th component of vector V;

[0105] S32. Use a vector matching algorithm (either cosine similarity algorithm or Euclidean distance-based similarity algorithm) to perform similarity retrieval in the vector database, calculate the similarity between the question text vector and the text vectors stored in the vector database, and select several candidate texts with high matching degree.

[0106] Specifically, the similarity of the user's question text vector with the text vectors stored in the vector database is used to select N texts with a high degree of matching, and these N texts are the candidate texts.

[0107] During the matching process, similarity calculation methods such as cosine similarity and Euclidean distance can be used, or an efficient Approximate Nearest Neighbor (ANN) search algorithm can be used to quickly retrieve text blocks that are similar to the query vector, thereby improving the accuracy and efficiency of data retrieval.

[0108] Similarity matching algorithm:

[0109] Cosine similarity is a widely used metric for calculating the similarity of text data. Its basic principle is to calculate the cosine of the angle between two vectors in a multidimensional space to measure their degree of similarity.

[0110] The formula for calculating cosine similarity is as follows:

[0111]

[0112] Where x and y represent two vectors, x i and y i T(x,y) represents the component of a vector in the i-th dimension, where n is the dimension of the vector, and T(x,y) represents the cosine similarity between vectors x and y.

[0113] Euclidean distance is the most intuitive method of measuring distance, derived from the formula for calculating the straight-line distance between two points in Euclidean space.

[0114] In two-dimensional space, the formula for calculating the distance between point A(x1, y1) and point B(x2, y2) is as follows:

[0115]

[0116] Where d represents the Euclidean distance between point A and point B, and (x1, y1) and (x2, y2) represent the coordinates of the two points respectively;

[0117] Euclidean distance can be used to measure the straight-line distance between two vectors, and is suitable for measuring the relative proximity between data points.

[0118] S4. Based on the large language model 2, the candidate texts are relevance evaluated and hierarchically labeled. The candidate texts are then filtered and sorted according to the labeling results to generate highly credible question-and-answer results.

[0119] The process of evaluating the relevance of candidate texts and performing hierarchical annotation based on the large language model 2, and then filtering and ranking the candidate texts according to the annotation results to generate highly reliable question-and-answer results includes the following steps:

[0120] S41. Use the Large Language Model 2 to perform relevance analysis on the candidate texts, and label the candidate texts with relevance based on the degree of matching between the candidate texts and the question texts.

[0121] Specifically, using the Large Language Model 2 (in this embodiment, the Large Language Model can be GPT, DeepSeek, Qwen, or Llama) to evaluate the relevance and perform hierarchical labeling on candidate texts includes: using the Large Language Model 2 to evaluate the relevance of the retrieved candidate texts, and labeling the candidate texts as "highly relevant", "partially relevant", or "irrelevant" based on their matching degree with the user's question, in order to optimize the subsequent filtering, sorting, and answer generation process and ensure the accuracy and validity of the input data;

[0122] Specifically, the user question is paired with each candidate text to form an input pair for a one-to-one matching evaluation. A large language model 2 is then invoked for semantic matching analysis, directly outputting the relevance annotation results of the candidate texts. The annotation categories include:

[0123] Highly relevant: When the large language model 2 determines that the candidate text has a high degree of semantic matching with the user's question and can be directly used to generate accurate answers, it is marked as "highly relevant".

[0124] Partially relevant: When the large language model 2 determines that the candidate text has a certain semantic relevance to the user's question, but the information completeness or accuracy is insufficient, it is labeled as "partially relevant".

[0125] Irrelevant: When the large language model 2 determines that the candidate text has a low degree of semantic matching with the user's question or has no direct reference value, it is marked as "irrelevant".

[0126] The expression for relevance evaluation and hierarchical annotation of candidate texts based on the Large Language Model 2 is as follows:

[0127] Relevance_Tag=LLM'(Prompt; Question; Candidate_Text; θ')

[0128] In the formula, Relevance_Tag represents the annotation result of the candidate text, which has three possible values: "highly relevant", "partially relevant" or "unrelevant", LLM′() represents the calculation of the Large Language Model 2, Prompt represents the prompt word, Question represents the question entered by the user, Candidate_Text represents a candidate text, and θ′ represents the parameters of the Large Language Model 2.

[0129] The aforementioned large language model two can employ architectures such as GPT, DeepSeek, Qwen, or Llama. During the relevance screening and ranking optimization of candidate texts, a model with a larger parameter scale (such as DeepSeek-V3) can be used to enhance the accuracy of relevance assessment and hierarchical annotation, thereby improving the accuracy of retrieval results and the reliability of generated results. Furthermore, in this embodiment, large language model one and large language model two can use different specific models or the same specific model; this embodiment does not impose any restrictions. For example, both large language model one and large language model two can use DeepSeek-V3, or large language model one can use DeepSeek-V3 and large language model two can use Qwen2-72B.

[0130] The above-mentioned prompt mainly guides the large language model 2 to perform relevance evaluation and hierarchical annotation of candidate texts. Below is an example of a prompt:

[0131] For the following questions and texts, determine whether they are "highly relevant", "partially relevant", or "irrelevant".

[0132] The correlation is output directly without explanation.

[0133] Question: {Question}

[0134] Text: {Candidate_Text};

[0135] Output:

[0136] This prompt is for reference only, and those skilled in the art can adjust it according to the actual situation of the Large Language Model 2.

[0137] The following is an example input and output:

[0138] Example input: For the following questions and texts, determine whether they are "highly relevant", "partially relevant", or "irrelevant".

[0139] The correlation is output directly without explanation.

[0140] Question: What are the opening hours of the school library on weekdays?

[0141] Text: The school library is open from 8:00 to 22:00 on weekdays.

[0142] Example output: Highly relevant;

[0143] S42. Based on the relevance annotation results of the candidate texts, filter the candidate texts to obtain reference texts, and sort the reference texts in order of relevance from high to low.

[0144] Specifically, based on the annotation results, a specific strategy is used to filter candidate texts to obtain reference texts. These reference texts are then sorted in descending order of relevance to ensure that highly relevant information is prioritized for input into the large language model 2, thereby generating highly reliable question-and-answer results. This includes: based on the relevance annotation results of the candidate texts, a pre-defined filtering and sorting mechanism is used to ensure that highly relevant information is prioritized for input into the large language model 2, thus improving the reliability and accuracy of the question-and-answer results. Specifically, if all candidate texts are marked as "irrelevant," then all candidate texts are input into the large language model 2 as reference texts to retain potentially effective information to the greatest extent possible, ensuring that the question-and-answer system can still provide a relatively reasonable answer even without directly matching text. Conversely, all candidate texts marked as "irrelevant" are filtered out, and the remaining candidate texts are used as reference texts. Texts marked as "highly relevant" and "partially relevant" among the reference texts are then input into the large language model 2 in descending order of relevance, with "highly relevant" being prioritized and "partially relevant" being secondary.

[0145] S43. Hint: In the second language model, the reference texts are input in descending order of relevance to generate highly reliable question-and-answer results.

[0146] Specifically, the large language model 2 is prompted that the reference texts are input in descending order of relevance to ensure that the generation process mainly references highly relevant information and improves the accuracy of the question-and-answer results.

[0147] Figure 4 An embodiment of the vertical domain intelligent question answering system based on hierarchical labeling and filtering sorting of the present invention is shown.

[0148] In this optional embodiment, the vertical domain intelligent question answering system based on hierarchical annotation and filtering sorting includes a vector database construction module, a question text data rewriting module, a retrieval and matching module, and a question answering result generation module;

[0149] The vector database construction module is used to acquire knowledge text data in the vertical domain and preprocess it, and then construct a vector database and index based on the preprocessed knowledge text data.

[0150] The question text data rewriting module is used to collect the question text data input by the user and rewrite the question text data using a large language model to optimize the expression.

[0151] The retrieval and matching module is used to obtain several candidate texts from the vector database based on the vector matching algorithm after the vectorized rewritten problem text data.

[0152] The question-and-answer result generation module is used to evaluate the relevance of candidate texts and perform hierarchical annotation based on the large language model 2, and to filter and sort the candidate texts according to the annotation results in order to generate highly credible question-and-answer results.

[0153] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores static and dynamic information data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the above method embodiment.

[0154] In addition, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0155] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0156] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0157] This invention is not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this invention is limited only by the appended claims.

Claims

1. A vertical field intelligent question answering method based on hierarchical labeling and screening ranking, characterized in that, The method comprises the following steps: S1, acquiring knowledge text data in a vertical field and preprocessing, constructing a vector database and an index based on the preprocessed knowledge text data; S2, collecting user input question text data, and using a large language model to rewrite the question text data to optimize the expression; The collection of user input question text data and the rewriting of the question text data by the large language model to optimize the expression comprises the following steps: S21, receiving original question text data input by a user, and using a preset prompt word template to splice the prompt word and the question text into a complete input sequence; S22, based on the prompt word guide, using a large language model to rewrite and optimize the input sequence after splicing to generate a rewritten text with complete semantics and in line with the specification; S3, vectorizing the rewritten question text data, and based on a vector matching algorithm, acquiring a plurality of candidate texts from the vector database; The vectorization of the rewritten question text data and the acquisition of a plurality of candidate texts from the vector database based on the vector matching algorithm comprise the following steps: S31, using a text embedding model to vectorize and encode the rewritten question text data to generate a corresponding high-dimensional vector representation, and normalizing the generated high-dimensional vector; S32, using a vector matching algorithm to perform similarity retrieval in the vector database, calculating the similarity between the question text vector and the stored text vector in the vector database, and selecting a plurality of high-matching-degree candidate texts; S4, based on a large language model, evaluating the relevance of the candidate texts and performing hierarchical labeling, and according to the labeling results, filtering and sorting the candidate texts to generate a high-confidence question and answer result; The relevance evaluation and hierarchical labeling of the candidate texts based on the large language model, and the filtering and sorting of the candidate texts according to the labeling results to generate a high-confidence question and answer result comprise the following steps: S41, using a large language model to analyze the relevance of the candidate texts, and labeling the relevance of the candidate texts according to the matching degree of the candidate texts and the question text; S42, filtering the candidate texts according to the relevance labeling results of the candidate texts to obtain reference texts, and sorting the reference texts in order from high to low according to the relevance; S43, prompting the large language model that the reference texts are input in order from high to low according to the relevance to generate a high-confidence question and answer result.

2. The vertical domain intelligent question answering method based on hierarchical labeling and screening sorting according to claim 1, characterized in that, The acquisition of knowledge text data in a vertical field and preprocessing, and the construction of a vector database and an index based on the preprocessed knowledge text data comprise the following steps: S11, acquiring knowledge text data in a vertical field, and performing document parsing and conversion from a non-text format to a pure text format; S12, using a semantic chunker to perform semantic-based chunking processing on the knowledge text data after format conversion to extract information units with independent semantics; S13, based on a text embedding model, performing vector conversion on the knowledge text blocks, and performing normalization processing on the converted vectors to eliminate vector scale differences; S14, a mapping relationship between the knowledge text block and its corresponding vector representation is established by using the index structure, and the index structure and the vectorization data are stored in the vector database to support efficient retrieval and similarity matching.

3. The vertical domain intelligent question answering method based on hierarchical labeling and screening sorting according to claim 2, characterized in that, The expression of the vectorization conversion is: ； The expression of the normalization processing is: ； ； In the formula, T i represents the input of the i-th knowledge text block, f() represents the encoding function of the text embedding model, V i represents the high-dimensional vector corresponding to the i-th knowledge text block, represents the normalized vector corresponding to the i-th knowledge text block, represents the Euclidean norm of the high-dimensional vector , V i，j represents the j-th component of the high-dimensional vector V i , and n represents the total number of knowledge text blocks.

4. The vertical domain intelligent question answering method based on hierarchical labeling and screening sorting according to claim 1, characterized in that, The expression of the problem text data rewriting process is: ； In the formula, represents the rewritten user question text, () represents the calculation of the large language model one, represents the prompt word, represents the original question text input by the user, represents the parameters of the large language model one.

5. The vertical domain intelligent question answering method based on hierarchical labeling and screening sorting according to claim 1, characterized in that, The vector matching algorithm includes any one of a cosine similarity algorithm or a similarity algorithm based on a Euclidean distance.

6. The vertical domain intelligent question answering method based on hierarchical labeling and screening sorting according to claim 1, characterized in that, The expression of the relevance evaluation and hierarchical labeling of the candidate text based on the large language model two is: ； In the formula, indicates the labeling result of the candidate text, indicates the calculation of the large language model two, indicates the prompt word, indicates the question input by the user, indicates a candidate text, indicates the parameters of the large language model two.

7. The vertical domain intelligent question answering system based on hierarchical labeling and screening ranking, for realizing the steps of the vertical domain intelligent question answering method based on hierarchical labeling and screening ranking in any one of claims 1-6, characterized in that, The system includes a vector database construction module, a problem text data rewriting module, a retrieval matching module, and a question and answer result generation module. The vector database construction module is configured to obtain knowledge text data of a vertical field and perform preprocessing, construct a vector database and an index based on the preprocessed knowledge text data. The problem text data rewriting module is configured to collect user input problem text data, and rewrite the problem text data by using a large language model one to optimize the expression. The retrieval matching module is configured to vectorize the rewritten problem text data, obtain a plurality of candidate texts from the vector database based on a vector matching algorithm. The question and answer result generation module is configured to evaluate the relevance of the candidate texts and perform hierarchical labeling based on a large language model two, and filter and sort the candidate texts according to the labeling results to generate a high-confidence question and answer result.

Images