Replay method, device, equipment and product combined with knowledge base

Abstract

The invention discloses a reply method and device combined with a knowledge base, equipment and a product, and relates to the technical field of intelligent question answering, and the method comprises the steps: receiving user question information, and analyzing the user question information to obtain a query label set; based on the query tag set, performing knowledge query through a tagged knowledge base to obtain a first candidate document set, and performing fusion rearrangement on the first candidate document set to obtain a document list; and inputting the document list into a large language model to obtain questions and answers. Therefore, the problem of poor reply effect caused by lack of multi-dimensional retrieval during question answering in the prior art is solved, and the accuracy of reply in combination with the knowledge base is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent question answering, and in particular to a reply method and device combining a knowledge base, equipment and products. BACKGROUND

[0002] With the development of large language models (LLM), the retrieval-augmented generation (RAG) based question answering system has become the core tool of enterprise knowledge management. The traditional large language model question answering effect depends on the pre-training knowledge base, but the knowledge base is static and cannot reflect the latest data and professional content in the enterprise. The RAG technology retrieves relevant information from the enterprise knowledge base before the model generates an answer, so that the model can refer to information in the vertical field during the generation process, thereby improving the relevance and accuracy of the output content, reducing the cost of retraining or continuing to train the large model, and injecting enterprise background knowledge into the answer.

[0003] However, the existing RAG system has significant defects in practical application. First, the construction of the knowledge classification system is difficult. The platform knowledge base is mostly unstructured documents, and the cost of manually constructing a unified classification system and labeling is high, low in efficiency and difficult to maintain, resulting in slow user positioning of target documents and irrelevant results returned by the system. Second, the metadata information is scarce. The uploaded documents often lack accurate metadata such as author and department, and the retrieval only relies on the similarity calculation of the document text, lacking multi-dimensional retrieval clues. In addition, the recall rate and precision of retrieval are insufficient. When there is a semantic gap between the user's query words and the document content, traditional vector retrieval is prone to failure. Not only that, the existing RAG system cannot support fine-grained and feature-oriented retrieval, and users cannot narrow down the search range by combining multiple label conditions, resulting in inflexible search experience.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide a reply method, device, equipment and product combining a knowledge base, aiming to solve the technical problem that the prior art lacks multi-dimensional retrieval when asking questions and replying, resulting in poor reply effect.

[0006] To achieve the above purpose, the present application provides a reply method combining a knowledge base, which comprises: receiving user question information, and analyzing the user question information to obtain a query label set; based on the query label set, performing knowledge query through a labeled knowledge base to obtain a first candidate document set, and performing fusion and rearrangement on the first candidate document set to obtain a document list; inputting the document list into a large language model to obtain a question reply.

[0007] In an embodiment, before the step of based on the query label set, performing knowledge query through a labeled knowledge base to obtain a first candidate document set, and performing fusion and rearrangement on the first candidate document set to obtain a document list, the method further comprises: performing analysis on the original knowledge document to obtain text content; performing segmentation on the text content according to semantic boundaries to obtain text segments; performing marking on the text segments to obtain text labels, and performing weight calculation according to the text labels to obtain label weights; performing vectorization on the text segments to obtain text vectors; storing the label weights and the text vectors into the labeled knowledge base.

[0008] In an embodiment, the step of analyzing the user question information to obtain a query label set comprises: performing retrieval on a global label library through the user question information to obtain an explicit intent; performing intent analysis on the user question information through a natural language extraction layer to obtain an implicit intent; labeling the explicit intent as a strong signal, labeling the implicit intent as a weak signal, and generating a query label set based on the labeled explicit intent and implicit intent.

[0009] In an embodiment, the step of based on the query label set, performing knowledge query through a labeled knowledge base to obtain a first candidate document set, and performing fusion and rearrangement on the first candidate document set to obtain a document list comprises: transforming the user question information into a high-dimensional vector through an embedding model; performing document selection on the labeled knowledge base through the high-dimensional vector to obtain a first candidate document set; based on the user question information and the query label set, performing fusion and rearrangement on the first candidate document set through a ranking model to obtain a second candidate document set; generating a document list based on the second candidate document set.

[0010] In an embodiment, the step of performing document selection on the labeled knowledge base through the high-dimensional vector to obtain a first candidate document set comprises: perform cosine similarity calculation on the document segments in the tagged knowledge base through the high-dimensional vector to obtain a document similarity, and perform document selection based on the document similarity to obtain a third candidate document set; perform matching on the document label set in the tagged knowledge base based on the query label set to obtain a fourth candidate document set; merge the third candidate document set and the fourth candidate document set to obtain a first candidate document set.

[0011] In an embodiment, the step of obtaining a second candidate document set by fusing and rearranging the first candidate document set based on the user question information and the query label set through a ranking model includes: performing cosine similarity calculation on the first candidate document set through the user question information to obtain a semantic similarity score; performing label matching degree calculation on the first candidate document set through the query label set to obtain a label matching score; performing label quality analysis on the query label set through the label weight of the first candidate document set to obtain a label quality score; fusing and rearranging the semantic similarity score, the label matching score, and the label quality score through a ranking model to obtain a second candidate document set.

[0012] In an embodiment, after the step of inputting the document list into a large language model to obtain a question reply, the method further includes: performing contribution degree calculation on the document list through a contribution degree calculation model based on the question reply to obtain a document contribution degree; receiving a feedback score of the user, and adjusting the label weight of the document list according to the feedback score and the document contribution degree to obtain an adjustment result; updating the tagged knowledge base based on the adjustment result to obtain an updated tagged knowledge base.

[0013] In addition, to achieve the above-mentioned purpose, the application further provides a reply device combined with a knowledge base, which includes: The parsing module is configured to receive user question information, and parse the user question information to obtain a query label set. The query module is configured to perform knowledge query through a tagged knowledge base based on the query label set to obtain a first candidate document set, and fuse and rearrange the first candidate document set to obtain a document list. The reply module is configured to input the document list into a large language model to obtain a question reply.

[0014] In addition, to achieve the above object, the application further provides a device for replying in combination with a knowledge base, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for replying in combination with a knowledge base as described above.

[0015] In addition, to achieve the above object, the application further provides a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the method for replying in combination with a knowledge base as described above.

[0016] In addition, to achieve the above object, the application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the method for replying in combination with a knowledge base as described above.

[0017] The one or more technical solutions provided by the application have at least the following technical effects: The method, device, equipment and product for replying in combination with a knowledge base provided by the embodiments of the application have the following technical effects: The drawings accompanying the specification illustrate the embodiments consistent with the application and together with the specification serve to explain the principles of the application.

[0018] The drawings accompanying the specification are incorporated into and form part of the specification, show embodiments consistent with the application, and together with the specification serve to explain the principles of the application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced as follows, and obviously, other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0020] Figure 1A flowchart provided by the reply method with the knowledge base in the first embodiment of the present application; Figure 2 A flowchart provided by the reply method with the knowledge base in the second embodiment of the present application; Figure 3 A brief flowchart of the reply method with the knowledge base in the second embodiment of the present application; Figure 4 A module structure diagram of the reply device with the knowledge base in the second embodiment of the present application; Figure 5 A device structure diagram of the hardware running environment related to the reply method with the knowledge base in the second embodiment of the present application.

[0021] The object, function features and advantages of the present application will be further explained with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not used to limit the present application.

[0023] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] The main solution of the embodiment of the application is: the original knowledge document is parsed to obtain text content; the text content is divided according to semantic boundaries to obtain text segments; the text segments are labeled to obtain text labels, and weight calculation is performed according to the text labels to obtain label weights; the text segments are vectorized to obtain text vectors; and the label weights and the text vectors are stored in the labeled knowledge base. The global label library is searched through the user question information to obtain an explicit intention; the user question information is analyzed through a natural language extraction layer to obtain an implicit intention; the explicit intention is marked as a strong signal, and the implicit intention is marked as a weak signal, and a query label set is generated based on the marked explicit intention and implicit intention. The user question information is converted into a high-dimensional vector through an embedding model; the labeled knowledge base is searched through the high-dimensional vector to obtain a first candidate document set; the first candidate document set is fused and rearranged through a ranking model based on the user question information and the query label set to obtain a second candidate document set; and a document list is generated based on the second candidate document set. The document segments in the labeled knowledge base are calculated for cosine similarity through the high-dimensional vector to obtain a document similarity, and a third candidate document set is obtained based on the document similarity; a document label set in the labeled knowledge base is matched based on the query label set to obtain a fourth candidate document set; and the third candidate document set and the fourth candidate document set are merged to obtain the first candidate document set. The first candidate document set is calculated for cosine similarity through the user question information to obtain a semantic similarity score; the first candidate document set is calculated for label matching degree through the query label set to obtain a label matching score; the query label set is analyzed for label quality through the label weight of the first candidate document set to obtain a label quality score; and the second candidate document set is obtained by fusing and rearranging the semantic similarity score, the label matching score, and the label quality score through a ranking model. Based on the question and answer, the document list is calculated for contribution degree through a contribution degree calculation model to obtain a document contribution degree; the feedback score of the user is received, and the label weight of the document list is adjusted according to the feedback score and the document contribution degree to obtain an adjustment result; and the labeled knowledge base is updated based on the adjustment result to obtain an updated labeled knowledge base. Thus, the problem that the prior art lacks multi-dimensional retrieval when answering questions, resulting in poor answering effect, is solved, the answering combined with the knowledge base is realized, and the accuracy of the answering combined with the knowledge base is improved.Based on the scheme, the problems of difficulty in constructing a knowledge classification system, lack of metadata information, insufficient recall and precision, and inability to support fine-grained and feature-oriented retrieval are solved, and the accuracy of the reply method combined with the knowledge base is improved.

[0025] In this embodiment, the reply device combined with the knowledge base is taken as the execution subject for the convenience of description.

[0026] Due to the limitations of the RAG system in enterprise knowledge management applications, the retrieval efficiency and user experience need to be improved. One of the problems is the difficulty in constructing a knowledge classification system. The platform knowledge base is composed of a large number of unstructured documents with diverse sources and complex topics. It is difficult and inefficient to manually construct a unified classification system, which will lead to a decrease in the reliability of the system and a decrease in the user experience. Another problem is the lack of metadata information. The uploaded documents often lack accurate metadata, which leads to the dependence of the retrieval system on the text for similarity calculation and the lack of multi-dimensional retrieval clues. In addition, the recall and precision of the retrieval are insufficient. When there is a semantic gap between the user's query and the document content, the traditional vector retrieval is prone to failure, which leads to insufficient recall and precision. Furthermore, the inability to support fine-grained retrieval leads to a lack of flexibility in the retrieval experience. Therefore, in the current enterprise knowledge management applications, there are difficulties in retrieval efficiency and user experience. Due to the above defects, if the RAG system fails to solve the problems of classification system, metadata, retrieval algorithm, and fine-grained retrieval, the system reliability and user experience will be reduced, and it will be difficult to meet the needs of precise knowledge management in enterprises.

[0027] The application provides a solution to analyze the received user question to obtain a query label set, and then perform knowledge query on the labeled knowledge base through the query label set to obtain a first candidate document set. Then, the first candidate document set is fused and rearranged to obtain a document list, which solves the problem of inability to perform multi-dimensional retrieval. Finally, the document list is input into a large language model to generate a question reply, which solves the problem of lack of multi-dimensional retrieval in the prior art when performing question reply, improves the accuracy of the reply combined with the knowledge base, and provides better services for users.

[0028] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a reply device combined with a knowledge base, etc. The reply device combined with a knowledge base is taken as an example to describe the embodiment and the following embodiments.

[0029] Based on this, the application embodiment provides a reply method combined with a knowledge base, referring to Figure 1 , Figure 1 The flowchart of the first embodiment of the reply method combined with a knowledge base of the application is shown.

[0030] In the embodiment, the reply method combined with a knowledge base includes steps S01-S03: Step S01, receiving user question information, and analyzing the user question information to obtain a query label set; Before the embodiment begins to be described, it should be clear that, with the development of large language models (LLM), the retrieval augmented generation (RAG) question and answer system has become a core tool for enterprise knowledge management. The traditional LLM relies on a static pre-trained knowledge base and cannot adapt to the latest data and professional content of the enterprise. The RAG retrieves the internal knowledge of the enterprise before generating an answer, which not only improves the relevance and accuracy of the output and reduces the cost of model training, but also injects enterprise background knowledge.

[0031] However, the existing RAG has obvious deficiencies. First, there are many unstructured documents, and the cost of manually constructing and maintaining a knowledge classification system is high and inefficient, affecting the reliability of retrieval. Second, the document metadata is lacking, and there is a lack of multi-dimensional retrieval clues. Third, the recall rate and precision of retrieval are insufficient, and matching failure is easy to occur due to semantic gap. Fourth, it does not support fine-grained and characteristic retrieval, and the retrieval experience is not flexible.

[0032] Therefore, in the embodiment, when the user question information is received, the user question information is analyzed to obtain a query label set. In the embodiment, the user question information received by the business system and the like refers to the business question of the user, for example, when the user is blocked during a transfer, the user will ask the business system to issue the user question information.

[0033] Step S02, based on the query label set, performing knowledge query through a labeled knowledge base to obtain a first candidate document set, and fusing and rearranging the first candidate document set to obtain a document list; After obtaining the query label set by analyzing the user question information, knowledge query is performed through the pre-obtained labeled knowledge base to obtain a first candidate document set. In the embodiment, the labeled knowledge base contains a plurality of original knowledge documents (such as PDF, Word, HTML), and the acquisition of multi-structured knowledge can be realized. Then, the first candidate document set is fused and rearranged to obtain a document list used for finally generating a reply.

[0034] In step S03, the document list is input into a large language model to obtain a question reply.

[0035] Finally, the document list is input into a large language model (LLM) to obtain a question reply used for finally replying to the user. The large language model (LLM) is an artificial intelligence model based on massive text data training and taking a Transformer as a core architecture. The core capability is to understand human language intention and generate natural language content conforming to grammar and logic. The large language model (LLM) covers multi-field general knowledge and vertical industry professional content (such as enterprise documents and industry rules), has strong context understanding capability, can link long text logic and accurately respond to complex queries, supports multiple languages and multiple tasks, and can complete diversified language-related tasks such as question answering, content creation, translation and logical reasoning.

[0036] Specifically, before the step S02 of obtaining a first candidate document set by performing knowledge query through a labeled knowledge base based on the query label set and fusing and rearranging the first candidate document set to obtain a document list, the method further includes the following steps. In step S0201, the original knowledge document is parsed to obtain text content. In step S0202, the text content is segmented according to a semantic boundary to obtain a text segment. In step S0203, the text segment is labeled to obtain a text label, and a label weight is calculated according to the text label. In step S0204, the text segment is vectorized to obtain a text vector. In step S0205, the label weight and the text vector are stored in the labeled knowledge base.

[0037] This module is the cornerstone of building a label ecosystem, which is responsible for parsing and blocking knowledge and generating a label set based on a text block using a plurality of natural language processing technologies. The processing flow is as follows: The original knowledge document in the embodiment refers to an original knowledge document such as a PDF, a Word or an HTML. The original knowledge document is parsed to extract pure text content in the document, thereby laying a foundation for subsequent processing.

[0038] The extracted pure text content is then segmented into appropriate size text segments according to semantic boundaries, facilitating subsequent label annotation and vectorization operations, and a label set (Ldoc) is generated for each text segment, including two types of label subsets, an authoritative label subset (Lauth) annotated by domain experts through a management interface with high-quality and high-level concept labels (such as "financial standards" and "V1.2 version"), and an automatically extracted label subset (Lauto) generated by various natural language processing techniques, including the use of TF-IDF and TextRank algorithms to extract key phrases, the use of NER models to identify names, places, times, and amounts of named entities, and the use of pre-trained language models to classify and label related labels based on a general label library.

[0039] Furthermore, the system maintains a dynamic weight w (l) for each label, with initial weights set according to label sources, with manually annotated label weights set to whigh, zero-shot classification and NER generated label weights set to wmedium, and key phrase extraction generated label weights set to wlow. The weight supports dynamic adjustment, and the weight of automatically generated labels is raised to whigh after being manually reviewed and confirmed. When a document is associated with a label and clicked or adopted by a user, the associated label weight will be slightly increased (wl=wl+ w). Then, the segmented text segments are vectorized to generate corresponding text vectors for subsequent semantic retrieval and matching.

[0040] Finally, the label set (including label source and label text), label weight, and text vector corresponding to the text segment are structured and stored together in the labeled knowledge base for subsequent retrieval.

[0041] More specifically, the step S01 of parsing the user query information to obtain a query label set includes: Step S011, searching the global label library through the user query information to obtain an explicit intent; Step S012, performing intent analysis on the user query information through a natural language extraction layer to obtain an implicit intent; Step S013, marking the explicit intent as a strong signal and the implicit intent as a weak signal, and generating a query label set based on the marked explicit intent and implicit intent.

[0042] When the user inputs the query information, the system takes the user's current input content as the context, searches the most relevant labels from the global label library (search methods include vector similarity matching or literal matching), and provides these labels as candidates for the user to select; the selected label by the user constitutes the explicit intent.

[0043] Subsequently, the user question information is analyzed for intent by a natural language extraction layer, and the same automatic extraction techniques as in the knowledge processing stage are used, including key phrase extraction (TF-IDF, TextRank), named entity recognition (NER), and pre-trained language model zero-shot classification, to extract potential labels from the user question and form an implicit intent.

[0044] The explicit intent obtained in the above step is marked as a strong signal (sstrong), the implicit intent obtained in the above step is marked as a weak signal (sweak), the marked explicit intent and implicit intent are integrated, and a structured query label set (Lquery) containing label text and signal strength is generated.

[0045] In the above scheme, the user question information is received, the query label set is obtained by analyzing the user question information, the knowledge base is queried based on the query label set, the first candidate document set is obtained, and the document list is obtained by fusing and rearranging the first candidate document set. The question reply is obtained by inputting the document list into the large language model. Thus, the received user question is analyzed to obtain a query label set, and then the knowledge base is queried based on the query label set to obtain a first candidate document set. The first candidate document set is fused and rearranged to obtain a document list, which solves the problem of being unable to perform multi-dimensional retrieval. Finally, the document list is input into the large language model to generate a question reply, which solves the problem of lacking multi-dimensional retrieval in the prior art when generating a question reply, resulting in poor reply effect, and improves the accuracy of the reply combined with the knowledge base.

[0046] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above first embodiment can be referred to the above introduction, and will not be described in detail. On this basis, please refer to Figure 2 , in step S02, based on the query label set, the knowledge base is queried to obtain a first candidate document set, and the first candidate document set is fused and rearranged to obtain a document list. The reply method combined with the knowledge base further includes steps S021-S024: Step S021, the user question information is converted into a high-dimensional vector by an embedding model; Step S022, the high-dimensional vector is used to select documents from the labeled knowledge base to obtain a first candidate document set; Step S023, based on the user question information and the query label set, the first candidate document set is fused and rearranged by a ranking model to obtain a second candidate document set; Step S024, a document list is generated based on the second candidate document set.

[0047] Firstly, the user question information (Qtext) is converted into a high-dimensional vector (Vq) by embedding model, providing a vector matching basis for subsequent semantic retrieval.

[0048] Then, the generated high-dimensional vector (Vq) is subjected to approximate nearest neighbor search in the vector database of the labeled knowledge base, the cosine similarity of the high-dimensional vector and all document segment vectors in the knowledge base is calculated, and the top-k documents with the highest similarity are selected to form an initial candidate set (Dvector), which is the first candidate document set.

[0049] Next, based on the user question information and the query label set, a ranking model is called to fuse and rearrange the first candidate document set, considering multiple dimensions such as semantic similarity between documents and questions, matching strength between document labels and query label set, quality weight of matching labels, etc. The document relevance score is calculated through the model, and the second candidate document set is obtained after filtering and optimization according to the score.

[0050] Finally, the second candidate document set is arranged in descending order of relevance score to generate a structured document list with clear structure and clear priority, providing accurate context support for subsequent large language model response generation.

[0051] Specifically, the step S022 of selecting documents from the labeled knowledge base based on the high-dimensional vector to obtain the first candidate document set comprises: Step S0221, calculate the cosine similarity of the high-dimensional vector and the document segment in the labeled knowledge base to obtain the document similarity, and select the documents based on the document similarity to obtain the third candidate document set; Step S0222, match the document label set in the labeled knowledge base based on the query label set to obtain the fourth candidate document set; Step S0223, merge the third candidate document set and the fourth candidate document set to obtain the first candidate document set.

[0052] First, the high-dimensional vector (Vq) converted by the user question information through the embedding model is used as the retrieval reference, and all document segment vectors stored in the labeled knowledge base (these vectors are stored after the text segment is vectorized in the knowledge processing stage, and correspond to the document segment) are called. The cosine similarity between the high-dimensional vector (Vq) and each document segment vector is calculated through the vector database tool. The cosine similarity is quantified by measuring the cosine value of the angle between two vectors, which quantifies the consistency of the direction of the vector in the semantic space (the closer the value is to 1, the more similar the semantics).

[0053] Then, all document segments are ranked in descending order according to the cosine similarity scores, and the top-k document segments with the highest similarity (k is a preset parameter, which can be dynamically adjusted according to the size of the knowledge base, such as k=50) are selected to form an initial candidate set (Dvector) containing only semantically relevant documents. This set is the third candidate document set.

[0054] Then, each document in the labeled knowledge base is traversed, and the complete label set (Ldoc) corresponding to the document is extracted. Ldoc is a structured tuple list [tag_id, tag_text, w (l)], where tag_id is the unique identification of the label, tag_text is the text content of the label, and w (l) is the weight of the label (reflecting the credibility of the label source, the weight of the manually labeled label is whigh, and the weight of the automatically generated label is wmedium or wlow).

[0055] Next, the query label set (Lquery, containing explicit strong signal labels and implicit weak signal labels, in the format [tag_id, tag_text, s], and s is the signal strength) is used as a reference to match the document label set (Ldoc) one by one. For each label in Lquery, an exact match (i.e., a label with the same tag_id or the same tag_text) is found in Ldoc. If a match is found, the signal strength (s, with strong signal sstrong taking a higher value than weak signal sweak) of the label is multiplied by the weight (w (l)) of the corresponding label in the document label set to obtain a single-label matching score. The total matching score of the document is obtained by accumulating the single-label matching scores of all matching items (score=Σs×w (l)).

[0056] At the same time, a preset threshold θ (θ is dynamically calibrated according to historical retrieval accuracy, such as θ=0.6) is set to filter out documents with a total matching score exceeding θ to form a candidate set (Dtag) containing only label-related documents. This set is the fourth candidate document set.

[0057] Finally, the union operation is performed on the third candidate document set (Dvector, semantically similar documents) and the fourth candidate document set (Dtag, label-matched documents), and all documents in both sets are retained, with duplicate documents removed (if a document exists in both sets, it is retained only once). The merged first candidate document set covers both semantically related and label-concept-matched documents, avoiding missing key documents due to semantic gaps (such as the user saying "reimbursement" while the document uses "cancellation") and preventing missing semantically similar documents due to missing labels, thereby comprehensively improving the recall rate of the retrieval and providing a more comprehensive candidate basis for subsequent fine-grained sorting.

[0058] More specifically, the step S023 of fusing and rearranging the first candidate document set based on the user query information and the query label set through the ranking model to obtain the second candidate document set comprises the following steps: In step S0231, cosine similarity calculation is performed on the first candidate document set through the user query information to obtain a semantic similarity score; In step S0232, label matching degree calculation is performed on the first candidate document set through the query label set to obtain a label matching score; In step S0233, label quality analysis is performed on the query label set through the label weight of the first candidate document set to obtain a label quality score; In step S0234, the semantic similarity score, the label matching score, and the label quality score are fused and rearranged through the ranking model to obtain the second candidate document set.

[0059] For each document (doc) in the first candidate document set (Dcandidate), the semantic similarity score (Semantic score) with the user query information is calculated through vector retrieval technology, specifically: the high-dimensional vector (Vq) of the user query information converted by the embedding model is called, and cosine similarity calculation is performed on the text segment vector corresponding to the document to obtain a score, which is directly used as the semantic similarity score, representing the closeness of the document and the query in the semantic space (the higher the score, the more semantic fit).

[0060] Subsequently, label matching degree calculation is performed on each document (doc) in the first candidate document set based on the query label set (Lquery) to obtain a label matching score (Tag match score). The calculation logic is consistent with the assignment score (score) in the label recall stage. Each label in the query label set (Lquery) is traversed, and an exact match item (tag_id or tag_text is the same) is found in the document label set (Ldoc). The signal strength (s) of the matched label is multiplied by the document label weight (w (l)) and then accumulated to obtain a total score, which is the label matching score, representing the matching strength of the document and the query in the label concept (the higher the score, the closer the label association).

[0061] For each document (doc) in the first candidate document set, a tag quality analysis is performed on the query tag set (Lquery) in combination with its tag weight, to obtain a tag quality score. Specifically, all document tags that successfully match the query tag set (i.e., the tags in the document tag set Ldoc that match Lquery) are extracted, and the average value of the weights (w (l)) of these matching tags is calculated. This average value is the tag quality score, which represents the reliability of the matching tags themselves (for example, when a high-weight tag annotated by a human is matched, the score is higher, indicating that the matching relationship is more reliable).

[0062] Finally, a multi-feature fusion ranking model is used to fine-tune the ranking of the first candidate document set, and the multi-dimensional relevance of the documents and the query is comprehensively evaluated. Therefore, the semantic similarity score, the tag matching score, and the tag quality score obtained in the above steps are integrated into a three-dimensional feature vector, which serves as the comprehensive feature representation of each document. The feature vector is input into a machine learning ranking model (such as LambdaRank or XGBoost ranking model), and the model predicts the final relevance score of each document to the user query based on the weight relationships learned from the training data. The first candidate document set is reordered in descending order based on the relevance score predicted by the model, and the top N documents (N is a preset parameter, such as N = 20) with the highest scores are selected to form the second candidate document set that has been fine-tuned and optimized, providing a core basis for generating accurate answers in the future.

[0063] Further, in the above embodiment, after the step of inputting the document list into the large language model to obtain the question and answer, the method further comprises: Step S04, based on the question and answer, a contribution degree calculation model is used to calculate the contribution degree of the document list, to obtain the document contribution degree; Step S05, receiving the feedback score of the user, and adjusting the label weight of the document list according to the feedback score and the document contribution degree, to obtain an adjustment result; Step S06, updating the labeled knowledge base based on the adjustment result, to obtain an updated labeled knowledge base.

[0064] This module is a closed-loop component for continuous self-optimization of the present application, and its purpose is to convert the implicit and explicit feedback of the user on the answer results into fine-tuned adjustments of the system internal parameters (mainly the label weight). In this way, the system can learn from actual interactions, so that the retrieval effect evolves with the increase in usage frequency, thereby realizing the transition from a "static knowledge base" to a "dynamic intelligent agent". The workflow of this module can be divided into three core steps: feedback signal collection, contribution degree analysis, and dynamic weight adjustment.

[0065] (1) Multi-dimensional feedback signal collection and quantization: The system needs to uniformly quantify different sources and different intensities of user behavior into computable feedback scores. The specific rules are shown in Table 1.

[0066]

[0067] Table 1 User behavior collection and quantization table (2) Document contribution analysis: The system needs to reasonably distribute the overall question and answer feedback to each source document. The present application adopts a contribution distribution model based on document ranking position.

[0068] Assumption: In the document list returned after fusion rearrangement, the higher the ranking of the document, the greater the contribution to the large model to generate answers.

[0069] (3) Contribution calculation model: For each document in , a contribution weight is assigned .

[0070] Use decay function for calculation, exponential decay: . Where is the ranking of the document in the list (starting from 1), and base is a decay factor between 0 and 1.

[0071] Normalize , so that , which means the relative proportion of contribution.

[0072] Dynamic adjustment algorithm for label weight The system will update the label weight of the relevant document incrementally according to the feedback score and document contribution.

[0073] Update target: for each document used in the question and answer session , update the weight of each label contained in it.

[0074] Update method:

[0075] : Global learning rate, a preset constant, used to control the overall step size of weight update, to prevent over-adjustment caused by single feedback.

[0076] : Feedback score corresponding to the feedback behavior. When the user behavior is positive feedback (e.g. likes, adoption), the weight of the label will be reduced; when the user behavior is negative feedback (e.g. dislikes), the weight of the label will be increased.

[0077] : Document Contribution degree in answer generation.

[0078] : Label importance indication function. This is an enhancement factor to distinguish the "plasticity" of different source labels. If the label is automatically extracted, then is 1; if the label is manually annotated, then is 0.1. This means that the weight of the authoritative label annotated by humans is relatively stable and not easily modified by a single feedback, but can still be fine-tuned.

[0079] In addition, to prevent the weight from increasing indefinitely or becoming negative, the embodiment sets a boundary [ , ]. After updating, if , it is truncated to ; if , it is truncated to .

[0080] Through the above process, the following key effects are achieved: Positive reinforcement cycle: When the answer quality is high and the user approves, the label weight of the core document on which the answer is generated will be improved. In future similar queries, these documents will have a higher probability of being retrieved and ranked in the front row due to the high label weight, so that high-quality answers are more likely to be generated again.

[0081] Negative correction mechanism: When the answer quality is poor and denied by the user, the label weight of the related document will be reduced, reducing their interference in subsequent retrieval.

[0082] System self-evolution: The entire system is no longer a static retrieval tool, but a dynamic ecosystem that can learn from user behavior and continuously optimize its internal knowledge organization.

[0083] In this embodiment, the updated labeled knowledge base will be used as the basis data for subsequent retrieval and intent understanding, realizing the closed loop of "user feedback → weight adjustment → knowledge base optimization → retrieval effect improvement". This closed loop can form three key effects, namely, the label weight of the core document is improved when the feedback is positive, it is easier to be retrieved in future similar queries, the label weight of the related document is reduced when the feedback is negative, reducing the interference in subsequent retrieval, and ultimately promoting the system to evolve from a static knowledge base to a dynamic intelligent agent that can continuously learn.

[0084] The embodiment converts the user question information into a high-dimensional vector through the above scheme, specifically through an embedding model; selects documents from the labeled knowledge base through the high-dimensional vector to obtain a first candidate document set; fuses and rearranges the first candidate document set through a ranking model based on the user question information and a query label set to obtain a second candidate document set; and generates a document list based on the second candidate document set. Thus, the received user question is analyzed to obtain a query label set, and then the labeled knowledge base is queried through the query label set to obtain a first candidate document set, and then the first candidate document set is fused and rearranged to obtain a document list, solving the problem that multi-dimensional retrieval cannot be performed. Finally, the document list is input into a large language model to generate a question reply, solving the problem that the prior art lacks multi-dimensional retrieval when performing a question reply, resulting in poor reply effect, and improving the accuracy of the reply combined with the knowledge base.

[0085] By way of example, in order to facilitate understanding of the implementation process of the reply combined with the knowledge base obtained after the above embodiment one, please refer to Figure 3 , Figure 3 A brief flowchart of a reply combined with a knowledge base is provided, specifically: The knowledge processing and labeling module is used to build a searchable labeled knowledge base, and the module is responsible for preprocessing and labeling the documents in the enterprise knowledge base to provide a structured basis for subsequent retrieval.

[0086] The knowledge processing and labeling module generates multi-dimensional labels through artificial labels (annotated by domain experts) + automatic extraction (key phrases, entity recognition, and model zero-shot classification) double paths for the input knowledge documents (including metadata), cuts the documents into text segments (slices) according to semantics, and performs vectorization processing on the segments. Finally, the “text segment vector + label set with weights (including artificial / automatic sources)” is stored in a vector model (the core carrier of the labeled knowledge base).

[0087] The query intention understanding module is used to analyze user requirements as labeled retrieval clues. When the user initiates a question, the module is responsible for mining explicit and implicit query intentions, preprocessing the user question, extracting implicit labels through automatic extraction technology (key phrases, entity recognition, etc.) homologous to the knowledge processing module, providing a label component for the user to select explicit labels from the global label library, and finally integrating the explicit (user selection) and implicit (automatic extraction) labels to generate a structured query label set, clearly expressing the labeled expression of the user's retrieval requirements.

[0088] The label-enhanced retrieval and rearrangement module is used to realize accurate knowledge recall and sorting. The module performs hybrid retrieval and fine rearrangement based on a query label set and a semantic vector. On one hand, after the user question is vectorized, semantic similarity retrieval is performed in the vector database (traditional semantic vector recall). On the other hand, the query label set is matched with the document label set in the labeled knowledge base (label matching recall). Then, the two kinds of recall results are combined (hybrid candidate set), and the candidate documents are fine rearranged through a sorting model of multi-feature fusion of semantic similarity score, label matching score and label quality score, to obtain the final sorted documents, which are injected into the large language model.

[0089] The user feedback and optimization cycle is used to drive the dynamic evolution of the system. The module iteratively adjusts the label weight through user feedback, realizes self-optimization of the system, collects feedback (such as adoption, likes, clicks on source documents, etc.) of the large language model response from the user, quantifies the feedback into scores, dynamically adjusts the label weight of the corresponding document (manual label fine-tuning and automatic label adjustment on demand) in combination with the contribution degree of the document in the sorted documents (the higher the ranking, the higher the contribution degree), and feeds back the adjusted label weight to the knowledge processing and labeling module described above to update the labeled knowledge base, so that the subsequent retrieval is more in line with the user's needs, forming a continuous evolution closed loop of “retrieval-feedback-optimization-retrieval again”.

[0090] The four modules described above are closely related. The knowledge processing module constructs the labeled knowledge, the intent understanding module transforms the user demand, the retrieval and rearrangement module realizes accurate knowledge recall, and the feedback module drives the system iteration. Finally, the large language model outputs the intelligent response that fits the enterprise scenario, and the knowledge system is continuously optimized with user interaction.

[0091] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the reply method combined with the knowledge base of the present application. More simple transformations based on this technical concept are within the protection scope of the present application.

[0092] The present application also provides a reply device combined with a knowledge base. Please refer to Figure 4 The reply device combined with the knowledge base comprises: The analysis module 10 is used to receive user question information and analyze the user question information to obtain a query label set; The query module 20 is used to perform knowledge query through the labeled knowledge base based on the query label set, obtain a first candidate document set, and perform fusion rearrangement on the first candidate document set to obtain a document list; The reply module 30 is used to input the document list into the large language model to obtain a question reply.

[0093] The answering device combined with a knowledge base provided in the present application adopts the answering method combined with a knowledge base in the above embodiment, and can solve the technical problem that the prior art lacks multi-dimensional search when answering questions, resulting in poor answering effect. Compared with the prior art, the answering device combined with a knowledge base provided in the present application has the same beneficial effects as the answering method combined with a knowledge base provided in the above embodiment, and other technical features in the answering device combined with a knowledge base are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0094] The present application provides an answering device combined with a knowledge base, which comprises at least one processor and a memory connected with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the answering method combined with a knowledge base in the above embodiment one.

[0095] Reference will be made to the following description of the embodiments of the present application, taken in conjunction with the accompanying drawings, in which Figure 5 which shows a structural schematic diagram of an answering device combined with a knowledge base suitable for implementing the embodiments of the present application. The answering device combined with a knowledge base in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 5 The answering device combined with a knowledge base shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0096] As Figure 5As shown, the knowledge base combined reply device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory 1002 or loaded from a storage device 1003 into a random access memory 1004. Various programs and data required for operation of the xxx device are also stored in the random access memory 1004. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other by a bus 1005. An input / output interface 1006 is also connected to the bus. Generally, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the knowledge base combined reply device to communicate wirelessly or by wire with other devices to exchange data. Although the knowledge base combined reply device is shown as having various systems, it should be understood that all of the shown systems are not required to be implemented or possessed. More or fewer systems can alternatively be implemented or possessed.

[0097] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0098] The knowledge base combined reply device provided by the present disclosure adopts the knowledge base combined reply method in the above embodiments, and can solve the technical problem that the prior art lacks multi-dimensional retrieval when replying to a question, resulting in poor reply effect. Compared with the prior art, the knowledge base combined reply device provided by the present disclosure has the same beneficial effects as the knowledge base combined reply method provided by the above embodiments, and other technical features in the knowledge base combined reply device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0099] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or combinations thereof, to achieve the various aspects of the disclosure. In the description above, specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0100] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any modifications or equivalents of the application should be construed as falling within the scope of the application. The scope of the application should be determined by the appended claims.

[0101] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the reply method in combination with a knowledge base in the above-described embodiments.

[0102] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the embodiments, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any suitable combination thereof.

[0103] The above-described computer readable storage medium can be contained in the reply device in combination with a knowledge base, or can exist separately and not be assembled into the reply device in combination with a knowledge base.

[0104] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the knowledge base combined reply device, the knowledge base combined reply device is caused to: receive user query information, parse the user query information to obtain a query label set; based on the query label set, perform knowledge query through a labeled knowledge base to obtain a first candidate document set, and perform fusion and rearrangement on the first candidate document set to obtain a document list; and input the document list into a large language model to obtain a query reply.

[0105] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0106] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0107] The modules involved in the embodiments of the present application can be implemented in a software manner or in a hardware manner. In some cases, the name of the module does not constitute a limitation on the module itself.

[0108] The readable storage medium provided by the application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer programs) for executing the above-mentioned reply method combined with the knowledge base, and can solve the technical problem that the prior art lacks multi-dimensional retrieval when replying to questions, resulting in poor reply effect. Compared with the prior art, the computer readable storage medium provided by the application has the same beneficial effects as the reply method combined with the knowledge base provided by the above-mentioned embodiments, and will not be repeated here.

[0109] The application further provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned reply method combined with the knowledge base.

[0110] The computer program product provided by the application can solve the technical problem that the prior art lacks multi-dimensional retrieval when replying to questions, resulting in poor reply effect. Compared with the prior art, the computer program product provided by the application has the same beneficial effects as the reply method combined with the knowledge base provided by the above-mentioned embodiments, and will not be repeated here.

[0111] The above-mentioned is only part of the embodiments of the application, and does not limit the patent scope of the application, and any equivalent structural transformation, direct / indirect application in other related technical fields made by using the content of the application specification and drawings under the technical concept of the application are included in the patent protection scope of the application.

Claims

1. A response method incorporating a knowledge base, characterized in that, The response method that incorporates a knowledge base includes: Receive user query information and parse the user query information to obtain a query tag set; Based on the query tag set, knowledge is queried through a tag-based knowledge base to obtain a first candidate document set, and the first candidate document set is merged and rearranged to obtain a document list; Input the document list into the large language model to obtain the query response.

2. The response method combining a knowledge base as described in claim 1, characterized in that, Before the step of performing knowledge query through a tagged knowledge base based on the query tag set to obtain a first candidate document set, and then merging and rearranging the first candidate document set to obtain a document list, the method further includes: The original knowledge document is parsed to obtain the text content; The text content is segmented according to semantic boundaries to obtain text fragments; The text fragment is tagged to obtain text labels, and weights are calculated based on the text labels to obtain label weights; The text fragment is vectorized to obtain a text vector; The label weights and text vectors are stored in the label-based knowledge base.

3. The response method combining a knowledge base as described in claim 1, characterized in that, The step of parsing the user query information to obtain the query tag set includes: The explicit intent is obtained by searching the global tag library using the user's question information. The implicit intent is obtained by performing intent analysis on the user's question information through a natural language extraction layer. The explicit intent is marked as a strong signal, the implicit intent is marked as a weak signal, and a query tag set is generated based on the marked explicit and implicit intents.

4. The response method combining a knowledge base as described in claim 1, characterized in that, The steps of performing knowledge queries based on the query tag set through a tag-based knowledge base to obtain a first candidate document set, and then fusing and rearranging the first candidate document set to obtain a document list include: The user-generated query information is transformed into a high-dimensional vector using an embedding model. The first candidate document set is obtained by selecting documents from the tagged knowledge base using the high-dimensional vector. Based on the user question information and the query tag set, the first candidate document set is merged and rearranged using a ranking model to obtain the second candidate document set. A document list is generated based on the second candidate document set.

5. The response method combining a knowledge base as described in claim 4, characterized in that, The step of selecting documents from the tagged knowledge base using the high-dimensional vector to obtain the first candidate document set includes: The document similarity is calculated by performing cosine similarity calculation on the document fragments in the tagged knowledge base using the high-dimensional vector, and the document similarity is then used to select documents to obtain a third candidate document set. Based on the query tag set, the document tag set in the tagged knowledge base is matched to obtain the fourth candidate document set; The third and fourth candidate document sets are merged to obtain the first candidate document set.

6. The response method combining a knowledge base as described in claim 4, characterized in that, The step of fusing and rearranging the first candidate document set based on the user query information and query tag set using a ranking model to obtain the second candidate document set includes: The semantic similarity score is obtained by calculating the cosine similarity of the first candidate document set based on the user's question information. The tag matching degree is calculated on the first candidate document set using the query tag set to obtain the tag matching score; Tag quality score is obtained by performing tag quality analysis on the query tag set using the tag weights of the first candidate document set. The semantic similarity score, tag matching score, and tag quality score are fused and rearranged using a ranking model to obtain the second candidate document set.

7. The response method combining a knowledge base as described in claim 1, characterized in that, After the step of inputting the document list into the large language model to obtain the question and answer, the method further includes: Based on the question and answer, the contribution of the document list is calculated using a contribution calculation model to obtain the document contribution score. Receive feedback ratings from the users, and adjust the tag weights of the document list based on the feedback ratings and document contribution, to obtain the adjustment result; The tagged knowledge base is updated based on the adjustment results to obtain the updated tagged knowledge base.

8. A response device incorporating a knowledge base, characterized in that, The response device that incorporates a knowledge base includes: The parsing module is used to receive user query information and parse the user query information to obtain a query tag set; The query module is used to perform knowledge queries based on the query tag set through a tag-based knowledge base to obtain a first candidate document set, and to merge and rearrange the first candidate document set to obtain a document list. The response module is used to input the document list into the large language model to obtain responses to questions.

9. A response device incorporating a knowledge base, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the response method incorporating a knowledge base as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the response method incorporating a knowledge base as described in any one of claims 1 to 7.

Images