Vertical domain intelligent question answering method and system based on hierarchical labeling and screening sorting

Abstract

The invention belongs to the technical field of intelligent questioning and answering, and discloses a vertical field intelligent questioning and answering method and system based on hierarchical labeling and screening sequencing, and the method comprises the steps: obtaining and preprocessing knowledge text data of a vertical field, and constructing a vector database and an index based on the preprocessed knowledge text data; rewriting the question text data by using the large language model I; vectorizing the rewritten problem text data, and obtaining a plurality of candidate texts from a vector database based on a vector matching algorithm; and performing correlation evaluation and hierarchical labeling on the candidate texts based on the large language model II, and screening and sorting the candidate texts according to a labeling result to generate a high-credibility question and answer result. By combining a large model technology, a retrieval enhancement generation technology and an intelligent screening and sorting mechanism, the accuracy of a retrieval result is remarkably improved, so that the reliability and interpretability of a question and answer result are improved, and the interaction experience of a user is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent question answering technology, and in particular to a vertical field intelligent question answering method and system based on hierarchical labeling and screening and sorting. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, intelligent question-answering systems have been widely used across various industries. Intelligent question-answering systems in vertical fields have also gradually become an important component of information services. In vertical fields, users often require efficient and accurate retrieval and answers when searching for relevant information in those fields. However, existing intelligent question-answering methods and systems in vertical fields still have certain limitations in handling complex queries and providing highly accurate answers. This results in low information acquisition efficiency, impacting user experience and system practicality.

[0003] Chinese patent publication number CN118194993A discloses a digital campus intelligent question-answering service method and system based on LLM, including: preprocessing campus data, and after the preprocessing is completed, segmenting the text data obtained based on the preprocessing, segmenting it into multiple text blocks, constructing an Embedding model, inputting the first text vector data into the Embedding model for vectorization to obtain second text vector data, which can reduce the dimension of the second text vector. When the user needs to make a query, the embedding vector corresponding to the first input question can be obtained, and based on similarity calculation, multiple matching answers that are most similar to the embedding vector can be matched from the second text vector data, and then the first input question and the multiple matching answers are input into the LLM model as context to output the answer text information.

[0004] This technical solution primarily matches multiple answers based on similarity, using the vector results of preprocessed campus data. These answers are then fed into the LLM model to generate the final answer. However, after obtaining multiple matching answers based on similarity calculations, the matching answers are not effectively screened, but are directly fed into the LLM model to generate the final answer. However, since the retrieved matching answers may contain a large amount of irrelevant or low-relevance information, this can reduce the accuracy of the answers generated by the large language model, affecting the reliability of the question-answering system. Summary of the Invention

[0005] The embodiments of the present invention provide a vertical field intelligent question answering method and system based on hierarchical labeling and screening and sorting to solve the problems of the above-mentioned technologies in the prior art.

[0006] The purpose of the present invention is to address the problems in the above-mentioned prior art and provide a vertical field intelligent question-answering method and system based on hierarchical labeling and screening and sorting, which can significantly improve the accuracy of retrieval results, thereby enhancing the reliability and interpretability of question-answering results and effectively improving the user interaction experience.

[0007] According to a first aspect of the present invention, a vertical field intelligent question answering method based on hierarchical labeling and screening and sorting is provided, comprising the following steps:

[0008] S1. Obtain and preprocess knowledge text data in vertical fields, and build a vector database and index based on the preprocessed knowledge text data;

[0009] S2. Collect the question text data input by the user and rewrite it using a large language model to optimize the expression;

[0010] S3, after vectorizing the rewritten question text data, obtain several candidate texts from the vector database based on the vector matching algorithm;

[0011] S4. Based on the large language model 2, the candidate texts are evaluated for relevance and hierarchically annotated, and the candidate texts are screened and sorted according to the annotation results to generate highly reliable question-and-answer results.

[0012] Furthermore, the acquisition of knowledge text data in a vertical field and preprocessing thereof, and the construction of a vector database and index based on the preprocessed knowledge text data include the following steps:

[0013] S11. Acquire knowledge text data in vertical fields, perform document parsing, and convert non-text formats into plain text formats;

[0014] S12, using a semantic chunker to perform semantic-based chunking processing on the knowledge text data after format conversion to extract information units with independent semantics;

[0015] S13. Based on the text embedding model, perform vector conversion on the knowledge text block and perform normalization on the converted vector to eliminate vector scale differences;

[0016] S14. Use the index structure to establish a mapping relationship between the knowledge text block and its corresponding vector representation, and store the index structure and vectorized data in a vector database to support efficient retrieval and similarity matching.

[0017] Furthermore, the expression of vectorized conversion is:

[0018] V i =f(T i )

[0019] The normalized expression is:

[0020]

[0021] Where, T irepresents the i-th knowledge text block of the input, f() represents the encoding function of the text embedding model, V i Represents the high-dimensional vector corresponding to the i-th knowledge text block, represents the normalized vector corresponding to the i-th knowledge text block, ||V i || represents a high-dimensional vector V i The Euclidean norm of V i，j Represents a high-dimensional vector V i The j-th component of , n represents the total number of knowledge text blocks.

[0022] Furthermore, the step of collecting the question text data input by the user and rewriting the question text data using the large language model to optimize the expression includes the following steps:

[0023] S21, receiving the original question text data input by the user, and using a preset prompt word template to splice the prompt word and the question text into a complete input sequence;

[0024] S22. Based on the prompt word guidance, a large language model is used to rewrite and optimize the concatenated input sequence to generate a semantically complete and standardized rewritten text.

[0025] Furthermore, the expression of the question text data rewriting process is:

[0026] Rewrite_Question=LLM(Prompt;Original_Question;θ)

[0027] Where Rewrite_Question represents the rewritten user question text, LLM() represents the calculation of the large language model, Prompt represents the prompt word, Original_Question represents the original question text entered by the user, and θ represents the parameters of the large language model.

[0028] Furthermore, the vectorized rewritten question text data is used to obtain a number of candidate texts from a vector database based on a vector matching algorithm, including the following steps:

[0029] S31. Use the text embedding model to vectorize the rewritten question text data, generate a corresponding high-dimensional vector representation, and normalize the generated high-dimensional vector;

[0030] S32. Use a vector matching algorithm to perform similarity search in the vector database, calculate the similarity between the question text vector and the text vector stored in the vector database, and screen out several candidate texts with high matching degrees.

[0031] Furthermore, the vector matching algorithm includes any one of a cosine similarity algorithm and a similarity algorithm based on Euclidean distance.

[0032] Furthermore, the relevance evaluation and hierarchical annotation of candidate texts based on the large language model 2, and screening and sorting the candidate texts according to the annotation results to generate high-credibility question-answering results include the following steps:

[0033] S41. Using the large language model 2, the candidate texts are subjected to relevance analysis, and the candidate texts are labeled with relevance based on the degree of matching between the candidate texts and the question text;

[0034] S42. Screen the candidate texts according to the relevance annotation results of the candidate texts to obtain reference texts, and sort the reference texts in descending order of relevance;

[0035] S43. Tip: For the second large language model, reference texts are input in descending order of relevance to generate high-confidence question-answering results.

[0036] Furthermore, the expression for relevance evaluation and hierarchical annotation of candidate texts based on the large language model 2 is:

[0037] Relevance_Tag=LLM'(Prompt; Question; Candidate_Text; θ')

[0038] Where Relevance_Tag represents the annotation result of the candidate text, LLM′() represents the calculation of the large language model 2, Prompt represents the prompt word, Question represents the question entered by the user, Candidate_Text represents a candidate text, and θ′ represents the parameters of the large language model 2.

[0039] According to a second aspect of an embodiment of the present invention, a vertical field intelligent question answering system based on hierarchical labeling and screening and sorting is provided, including a vector database construction module, a question text data rewriting module, a retrieval and matching module, and a question and answer result generation module;

[0040] The vector database construction module is used to obtain and preprocess the knowledge text data in the vertical field, and to construct a vector database and index based on the preprocessed knowledge text data;

[0041] The question text data rewriting module is used to collect question text data input by the user and rewrite the question text data using a large language model to optimize the expression;

[0042] The retrieval and matching module is used to vectorize the rewritten question text data and obtain several candidate texts from the vector database based on the vector matching algorithm;

[0043] The question-answer result generation module is used to perform relevance evaluation and hierarchical annotation on candidate texts based on the large language model 2, and to screen and sort the candidate texts according to the annotation results to generate highly reliable question-answer results.

[0044] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0045] This invention introduces retrieval-enhanced generation technology, which can retrieve and obtain the latest information from external knowledge bases in real time. By rewriting user question texts using a large language model (1) and corresponding prompt words, the expression is optimized and the accuracy of the search results is improved. A large language model (2) is used to evaluate the relevance and hierarchically annotate candidate texts. Based on the annotated results, the candidate texts are screened and sorted, prioritizing highly relevant information to generate highly reliable question-and-answer results. Therefore, this invention can significantly enhance the reliability and interpretability of question-and-answer results, effectively improving the user's interactive experience.

[0046] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0048] Figure 1 This is a flowchart of a vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to an exemplary embodiment;

[0049] Figure 2 This is a schematic diagram illustrating a vertical field intelligent question-answering method based on hierarchical labeling and screening and sorting according to an exemplary embodiment;

[0050] Figure 3 This is a detailed architecture diagram of a vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to an exemplary embodiment;

[0051] Figure 4 It is a structural block diagram of a vertical field intelligent question-answering system based on hierarchical labeling and screening and sorting according to an exemplary embodiment. DETAILED DESCRIPTION

[0052] The following description and accompanying drawings sufficiently illustrate the specific embodiments herein to enable those skilled in the art to practice them. Portions and features of some embodiments may be included in or substituted for portions and features of other embodiments. The scope of the embodiments herein includes the entire scope of the claims, including all available equivalents thereof. Herein, the terms "first," "second," and the like are used solely to distinguish one element from another and do not require or imply any actual relationship or order between these elements. In practice, the first element can also be referred to as the second element, and vice versa. Furthermore, the terms "comprise," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a structure, device, or apparatus comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such structure, device, or apparatus. Without further limitation, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the structure, device, or apparatus comprising the element. The various embodiments herein are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Similar or identical parts between the various embodiments can be referenced to each other.

[0053] As used herein, unless otherwise specified, the term "plurality" means two or more.

[0054] In this document, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0055] In this article, the term "and / or" is used to describe the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0056] It should be understood that, although the various steps in the flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0057] Each module in the device or system of the present application can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software so that the processor can call and execute the operations corresponding to the above modules.

[0058] The relevant technical terms in the present invention are explained as follows:

[0059] Description of the Retrieval-Augmented Generation method (RAG): Retrieval-Augmented Generation (RAG) is a natural language processing technique that integrates information retrieval and text generation. The RAG method generally includes two core stages: first, using retrieval techniques such as vector search or keyword matching to obtain documents or fragments that are highly relevant to the user query from an external knowledge base; second, the retrieved content is input into the text generation model as context information to generate an output that better meets user needs and has more comprehensive information. This method introduces an external knowledge base or document storage into the process of generating content by a generative model (such as a large language model) to retrieve relevant information to enhance the accuracy and reliability of text generation.

[0060] A note about large language models (LLMs): Large language models (LLMs) are a type of natural language processing (NLP) model based on deep learning. They typically use large-scale neural network architectures (such as Transformers) and are pre-trained on massive amounts of text data to learn the syntax, semantics, and contextual relationships of a language. The core advantage of large language models lies in their powerful contextual understanding and generalization capabilities. These models generate high-dimensional word vector representations through autoregression or autoencoding, and are capable of performing a variety of language understanding and generation tasks, such as text generation, machine translation, question-answering systems, and summary generation.

[0061] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0062] Figure 1-Figure 3 An embodiment of the vertical field intelligent question answering method based on hierarchical labeling and screening and sorting of the present invention is shown.

[0063] In this optional embodiment, the vertical field intelligent question answering method based on hierarchical labeling and screening and sorting has the following steps:

[0064] S1. Obtain and preprocess knowledge text data in vertical fields, and build a vector database and index based on the preprocessed knowledge text data;

[0065] The process of obtaining and preprocessing knowledge text data in a vertical field, and constructing a vector database and index based on the preprocessed knowledge text data includes the following steps:

[0066] S11. Acquire knowledge text data in vertical fields, parse documents, and convert non-text formats into plain text formats to extract plain text content;

[0067] Specifically, the document type uploaded by the user is identified based on the document extension, and the corresponding loader is used to parse and process it. If the document type is JSON, the JSONReader in the LlamaIndex framework is used to parse it and store the processed data; if the document type is WORD, PDF or plain text format, the SimpleDirectoryReader loader provided by LlamaIndex is used to parse it and store the corresponding text data;

[0068] S12, using a semantic chunker to perform semantic-based chunking processing on the knowledge text data after format conversion to extract information units with independent semantics;

[0069] The semantic splitter (SemanticSplitterNodeParser) provided by LlamaIndex is used to perform semantic-based chunking on the text to identify and extract information units with independent semantics, thereby improving the semantic integrity of the text block and the accuracy of retrieval matching;

[0070] LlamaIndex adopts an embedding-based semantic chunking strategy. Its core mechanism is to calculate text similarity based on a sliding window and merge adjacent sentences whose similarity reaches a set threshold into the same text chunk, ensuring that content with strong semantic relevance is divided into the same unit, thereby ensuring the semantic integrity of the text chunk.

[0071] During the similarity calculation process, an embedding model is required for text vectorization. Optional models include BERT, RoBERTa, or GPT. Those skilled in the art can select a suitable embedding model based on specific application requirements and computing resources.

[0072] S13. Based on the text embedding model, perform vector conversion on the knowledge text block and perform normalization on the converted vector to eliminate vector scale differences;

[0073] Specifically, the text embedding model is used to vectorize the text block. The specific process is as follows: the text block is input into the text embedding model, and the model outputs the corresponding high-dimensional vector after calculation. The specific expression is as follows:

[0074] V i =f(Ti )

[0075] Where, T i represents the i-th knowledge text block of the input, f() represents the encoding function of the text embedding model, V i Represents the high-dimensional vector corresponding to the i-th knowledge text block;

[0076] Specifically, a normalization operation is performed on the generated vector. Given the generated vector, the normalization calculation formula is:

[0077]

[0078] Where, represents the normalized vector corresponding to the i-th knowledge text block, ||V i || represents a high-dimensional vector V i The Euclidean norm (L2 norm) of V i，j Represents a high-dimensional vector V i The j-th component of , n represents the total number of knowledge text blocks.

[0079] S14. Using the index structure to establish a mapping relationship between the knowledge text block and its corresponding vector representation, and storing the index structure and vectorized data in a vector database to support efficient retrieval and similarity matching;

[0080] Specifically, an existing text embedding model is used to transform each text block into a vector. The transformed vector data is then normalized to eliminate vector scale differences, thereby improving the accuracy of vector similarity calculations. Subsequently, the text blocks and their corresponding vector data are stored in a vector database and indexed to support efficient retrieval and matching operations.

[0081] S2. Collect the question text data input by the user and rewrite it using a large language model to optimize the expression;

[0082] When a user asks a question, receiving a question-answering request input by the user and preprocessing the input text includes: receiving the original question text input by the user and preprocessing it, wherein the preprocessing includes rewriting the user question text using a large language model and corresponding prompt words to optimize the expression and improve the accuracy of retrieval and answer generation;

[0083] The step of collecting the question text data input by the user and rewriting the question text data using a large language model to optimize the expression includes the following steps:

[0084] S21, receiving the original question text data input by the user, and using a preset prompt word template to splice the prompt word and the question text into a complete input sequence;

[0085] S22. Based on the prompt word guidance, a large language model is used to rewrite and optimize the concatenated input sequence to generate a semantically complete and standardized rewritten text.

[0086] Specifically, the concatenated input sequence is fed into the large language model for calculation. The model rewrites and optimizes the user's question text based on the prompt words, generating a semantically complete and standardized rewritten text. The rewritten text generated by the large language model is extracted and provided as the standardized question text to the subsequent vectorization processing and retrieval modules.

[0087] The expression of the problem text data rewriting process is:

[0088] Rewrite_Question=LLM(Prompt;Original_Question;θ)

[0089] Where Rewrite_Question represents the rewritten user question text, LLM() represents the calculation of the large language model, Prompt represents the prompt word, Original_Question represents the original question text entered by the user, and θ represents the parameters of the large language model.

[0090] The above prompt mainly prompts the large language model to rewrite the user's question text. Here is an example of a prompt:

[0091] Please rewrite the following user question text to optimize the expression and improve the accuracy of retrieval and answer generation. Please ensure that the semantics are consistent and eliminate ambiguity as much as possible.

[0092] User question text: {Original_Question};

[0093] Rewritten question text:

[0094] The rewritten question text should be clearer, more precise, and conform to the norms of natural language expression.

[0095] This prompt is for reference only, and those skilled in the art may adjust it according to the actual situation of the large language model 1.

[0096] S3, after vectorizing the rewritten question text data, obtain several candidate texts from the vector database based on the vector matching algorithm;

[0097] The vectorized rewritten question text data is then used to obtain a number of candidate texts from a vector database based on a vector matching algorithm, which includes the following steps:

[0098] S31. Use a text embedding model (the text embedding model in this embodiment can use BERT, RoBERTa, or GPT model, but needs to be consistent with the previous text embedding model) to vectorize the rewritten question text data, generate a corresponding high-dimensional vector representation, and normalize the generated high-dimensional vector;

[0099] Specifically, a text embedding model is used to vectorize the preprocessed user question text and generate a corresponding high-dimensional vector representation. The specific process is as follows: the preprocessed user question text is input into the text embedding model, and the model calculates and outputs the corresponding high-dimensional vector. The specific expression is:

[0100] V=f(T)

[0101] Where T represents the preprocessed user question text input, f() represents the encoding function of the text embedding model, and V represents the high-dimensional vector corresponding to the preprocessed user question text;

[0102] Specifically, a normalization operation is performed on the generated vector. Given the generated vector V, its normalization calculation formula is:

[0103]

[0104] Where V norm Represents the normalized vector corresponding to vector V, V represents the high-dimensional vector corresponding to the preprocessed user question text, ||V|| represents the Euclidean norm (L2 norm) of vector V, V j represents the jth component of vector V;

[0105] S32, using a vector matching algorithm (the vector matching algorithm includes any one of a cosine similarity algorithm and a similarity algorithm based on Euclidean distance) to perform similarity search in the vector database, calculate the similarity between the question text vector and the text vectors stored in the vector database, and screen out several candidate texts with high matching degrees;

[0106] Specifically, the user question text vector is used to perform similarity matching with the text vectors stored in the vector database, and N texts with a higher degree of matching are selected. These N texts are candidate texts.

[0107] During the matching process, similarity calculation methods such as cosine similarity and Euclidean distance, or efficient approximate nearest neighbor (ANN) search algorithms can be used to quickly retrieve text blocks that are relatively similar to the query vector, thereby improving the accuracy and efficiency of data retrieval.

[0108] Similarity matching algorithm:

[0109] Cosine similarity is a measurement method widely used in text data similarity calculation. Its basic principle is to calculate the cosine value of the angle between two vectors in multidimensional space to measure their similarity.

[0110] The calculation formula of cosine similarity is as follows:

[0111]

[0112] Among them, x and y represent two vectors, x i and y i represents the component of the vector in the i-th dimension, n is the dimension of the vector, and T(x,y) represents the cosine similarity between vectors x and y;

[0113] Euclidean distance is the most intuitive distance measurement method, which is derived from the formula for calculating the straight-line distance between two points in Euclidean space.

[0114] In two-dimensional space, the distance between point A (x1, y1) and point B (x2, y2) is calculated as follows:

[0115]

[0116] Where d represents the Euclidean distance between point A and point B, (x1, y1) and (x2, y2) represent the coordinates of the two points respectively;

[0117] Euclidean distance can be used to measure the straight-line distance between two vectors and is suitable for measuring the relative proximity between data points.

[0118] S4. Based on the large language model 2, the candidate texts are evaluated for relevance and hierarchically annotated, and the candidate texts are screened and sorted according to the annotation results to generate highly reliable question-and-answer results.

[0119] The process of performing relevance assessment and hierarchical annotation on candidate texts based on the large language model 2, and screening and sorting the candidate texts according to the annotation results to generate highly reliable question-answering results includes the following steps:

[0120] S41. Using the large language model 2, the candidate texts are subjected to relevance analysis, and the candidate texts are labeled with relevance based on the degree of matching between the candidate texts and the question text;

[0121] Specifically, using the large language model 2 (in this embodiment, the large language model can use the GPT, DeepSeek, Qwen or Llama model) to perform relevance evaluation and hierarchical annotation on the candidate texts includes: using the large language model 2 to evaluate the relevance of the retrieved candidate texts, and annotating the candidate texts into three categories of "highly relevant", "partially relevant" or "irrelevant" based on the degree of match with the user question, so as to optimize the subsequent screening, sorting and answer generation process and ensure the accuracy and validity of the input data;

[0122] Specifically, the user question and each candidate text are combined into an input pair for a one-by-one matching evaluation. The large language model 2 is called to perform semantic matching analysis and directly output the relevance annotation results of the candidate text. The annotation categories include:

[0123] Highly relevant: When the large language model determines that the candidate text has a high degree of semantic match with the user question and can be directly used to generate an accurate answer, it is marked as "highly relevant".

[0124] Partial relevance: When the large language model 2 determines that the candidate text has a certain degree of semantic relevance to the user question, but the information is insufficiently complete or accurate, it is marked as "partially relevant."

[0125] Irrelevant: When the large language model 2 determines that the semantic match between the candidate text and the user question is low or has no direct reference value, it is marked as "Irrelevant".

[0126] The expression for relevance evaluation and hierarchical annotation of candidate texts based on the large language model 2 is:

[0127] Relevance_Tag=LLM'(Prompt; Question; Candidate_Text; θ')

[0128] Where Relevance_Tag represents the annotation result of the candidate text, which can take three values: "highly relevant", "partially relevant", or "not relevant". LLM′() represents the calculation of the large language model 2. Prompt represents the prompt word, Question represents the question entered by the user, Candidate_Text represents a candidate text, and θ′ represents the parameters of the large language model 2.

[0129] The above-mentioned large language model 2 can use architectures such as GPT, DeepSeek, Qwen or Llama. Among them, in the process of relevance screening and ranking optimization of candidate texts, a model with a larger parameter scale (such as DeepSeek-V3) can be used to enhance the accuracy of relevance evaluation and hierarchical annotation, thereby improving the accuracy of retrieval results and the reliability of generated results. In addition, the large language model 1 and the large language model 2 in this embodiment can use different specific models or the same specific model. This is not limited in this embodiment. For example, both the large language model 1 and the large language model 2 can use DeepSeek-V3, or the large language model 1 can use DeepSeek-V3 and the large language model 2 can use Qwen2-72B.

[0130] The above prompt mainly prompts the large language model to perform relevance evaluation and hierarchical annotation on the candidate text. The following is an example of a prompt:

[0131] For the following questions and texts, decide whether they are “highly relevant,” “partially relevant,” or “not relevant.”

[0132] Output correlations directly without interpretation.

[0133] Question: {Question};

[0134] Text: {Candidate_Text};

[0135] Output:

[0136] This prompt is for reference only, and those skilled in the art may adjust it according to the actual situation of the large language model 2.

[0137] Here is a sample input and output:

[0138] Sample input: Given the following question and text, determine whether they are "highly relevant", "partially relevant", or "not relevant".

[0139] Output correlations directly without interpretation.

[0140] Question: What are the opening hours of the school library on weekdays?

[0141] Text: The school library is open from 8:00 to 22:00 every day on weekdays.

[0142] Sample output: Highly correlated;

[0143] S42. Screen the candidate texts according to the relevance annotation results of the candidate texts to obtain reference texts, and sort the reference texts in descending order of relevance;

[0144] Specifically, based on the annotation results, a specific strategy is used to filter candidate texts to obtain reference texts, and the reference texts are sorted in descending order of relevance to ensure that highly relevant information is input into the large language model 2 first, so as to generate highly reliable question-answering results. This includes: based on the relevance annotation results of the candidate texts, a preset screening and sorting mechanism is used to ensure that highly relevant information is input into the large language model 2 first, so as to improve the reliability and accuracy of the question-answering results. Specifically, if all candidate texts are marked as "irrelevant", all candidate texts are input into the large language model 2 as reference texts to retain potential effective information to the greatest extent possible, ensuring that the question-answering system can still provide relatively reasonable answers when there is no directly matching text; otherwise, all candidate texts marked as "irrelevant" are filtered out, and the remaining candidate texts are used as reference texts, and the texts marked as "highly relevant" and "partially relevant" in the reference texts are input into the large language model 2 in the order of "highly relevant" first and "partially relevant" second;

[0145] S43, prompting the large language model 2, the reference texts are input in descending order of relevance to generate high-confidence question-answering results;

[0146] Specifically, the large language model 2 is prompted that the reference texts are input in order of relevance from high to low, so as to ensure that the generation process mainly refers to high-relevance information and improve the accuracy of the question-answering results.

[0147] Figure 4 An embodiment of the vertical field intelligent question answering system based on hierarchical labeling and screening and sorting of the present invention is shown.

[0148] In this optional embodiment, the vertical field intelligent question answering system based on hierarchical labeling and screening and sorting includes a vector database construction module, a question text data rewriting module, a retrieval and matching module, and a question and answer result generation module;

[0149] The vector database construction module is used to obtain and preprocess the knowledge text data in the vertical field, and to construct a vector database and index based on the preprocessed knowledge text data;

[0150] The question text data rewriting module is used to collect question text data input by the user and rewrite the question text data using a large language model to optimize the expression;

[0151] The retrieval and matching module is used to vectorize the rewritten question text data and obtain several candidate texts from the vector database based on the vector matching algorithm;

[0152] The question-answer result generation module is used to perform relevance evaluation and hierarchical annotation on candidate texts based on the large language model 2, and to screen and sort the candidate texts according to the annotation results to generate highly reliable question-answer results.

[0153] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store static information and dynamic information data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment are implemented.

[0154] In addition, the present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiment when executing the computer program.

[0155] In addition, the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0156] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0157] The present invention is not limited to the structures described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A vertical field intelligent question answering method based on hierarchical labeling and screening and sorting, characterized by: The following steps are involved: S1. Obtain and preprocess knowledge text data in vertical fields, and build a vector database and index based on the preprocessed knowledge text data; S2. Collect the question text data input by the user and rewrite it using a large language model to optimize the expression; S3, after vectorizing the rewritten question text data, obtain several candidate texts from the vector database based on the vector matching algorithm; S4. Based on the large language model 2, the candidate texts are evaluated for relevance and hierarchically annotated, and the candidate texts are screened and sorted according to the annotation results to generate highly reliable question-and-answer results.

2. The vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to claim 1 is characterized in that: The acquisition of vertical field knowledge text data and preprocessing, and the construction of a vector database and index based on the preprocessed knowledge text data include the following steps: S11. Acquire knowledge text data in vertical fields, perform document parsing, and convert non-text formats into plain text formats; S12, using a semantic chunker to perform semantic-based chunking processing on the knowledge text data after format conversion to extract information units with independent semantics; S13. Based on the text embedding model, perform vector conversion on the knowledge text block and perform normalization on the converted vector to eliminate vector scale differences; S14. Use the index structure to establish a mapping relationship between the knowledge text block and its corresponding vector representation, and store the index structure and vectorized data in a vector database to support efficient retrieval and similarity matching.

3. The vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to claim 2 is characterized in that: The expression for vectorized conversion is: V i =f(T i ) The normalized expression is: Where, T i represents the i-th knowledge text block of the input, f() represents the encoding function of the text embedding model, V i Represents the high-dimensional vector corresponding to the i-th knowledge text block, represents the normalized vector corresponding to the i-th knowledge text block, ||V i || represents a high-dimensional vector V i The Euclidean norm of V i，j Represents a high-dimensional vector V i The j-th component of , n represents the total number of knowledge text blocks.

4. The vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to claim 1 is characterized in that: The step of collecting question text data input by the user and rewriting the question text data using a large language model to optimize the expression includes the following steps: S21, receiving the original question text data input by the user, and using a preset prompt word template to splice the prompt word and the question text into a complete input sequence; S22. Based on the prompt word guidance, a large language model is used to rewrite and optimize the concatenated input sequence to generate a semantically complete and standardized rewritten text.

5. The vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to claim 4 is characterized in that: The expression of the problem text data rewriting process is: Rewrite_Question=LLM(Prompt;Original_Question;θ) Where Rewrite_Question represents the rewritten user question text, LLM() represents the calculation of the large language model, Prompt represents the prompt word, Original_Question represents the original question text entered by the user, and θ represents the parameters of the large language model.

6. The vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to claim 1 is characterized in that: The vectorized rewritten question text data is used to obtain a number of candidate texts from a vector database based on a vector matching algorithm, including the following steps: S31. Use the text embedding model to vectorize the rewritten question text data, generate a corresponding high-dimensional vector representation, and normalize the generated high-dimensional vector; S32. Use a vector matching algorithm to perform similarity search in the vector database, calculate the similarity between the question text vector and the text vector stored in the vector database, and screen out several candidate texts with high matching degrees.

7. The vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to claim 6 is characterized in that: The vector matching algorithm includes any one of a cosine similarity algorithm and a similarity algorithm based on Euclidean distance.

8. The vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to claim 1 is characterized in that: The process of performing relevance assessment and hierarchical annotation on candidate texts based on the large language model 2, and screening and sorting the candidate texts according to the annotation results to generate highly reliable question-answering results includes the following steps: S41. Using the large language model 2, the candidate texts are subjected to relevance analysis, and the candidate texts are labeled with relevance based on the degree of matching between the candidate texts and the question text; S42. Screen the candidate texts according to the relevance annotation results of the candidate texts to obtain reference texts, and sort the reference texts in descending order of relevance; S43. Tip: For the second large language model, reference texts are input in descending order of relevance to generate high-confidence question-answering results.

9. The vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to claim 8 is characterized in that: The expression for relevance evaluation and hierarchical annotation of candidate texts based on the large language model 2 is: Relevance_Tag=LLM′(Prompt; Question; Candidate_Text; θ′) Where Relevance_Tag represents the annotation result of the candidate text, LLM′() represents the calculation of the large language model 2, Prompt represents the prompt word, Question represents the question entered by the user, Candidate_Text represents a candidate text, and θ′ represents the parameters of the large language model 2.

10. A vertical field intelligent question answering system based on hierarchical labeling and screening and sorting, used to implement the steps of the vertical field intelligent question answering method based on hierarchical labeling and screening and sorting according to any one of claims 1 to 9, characterized in that: The system includes a vector database construction module, a question text data rewriting module, a retrieval and matching module, and a question-answering result generation module; The vector database construction module is used to obtain and preprocess the knowledge text data in the vertical field, and to construct a vector database and index based on the preprocessed knowledge text data; The question text data rewriting module is used to collect question text data input by the user and rewrite the question text data using a large language model to optimize the expression; The retrieval and matching module is used to vectorize the rewritten question text data and obtain several candidate texts from the vector database based on the vector matching algorithm; The question-answer result generation module is used to perform relevance evaluation and hierarchical annotation on candidate texts based on the large language model 2, and to screen and sort the candidate texts according to the annotation results to generate highly reliable question-answer results.

Images