An enterprise knowledge question answering system, method, device, and medium

Abstract

This application relates to an enterprise knowledge question-answering system, method, device, and medium. The enterprise knowledge question-answering system includes a knowledge base module, a knowledge retrieval module, an answer generation module, and a user feedback module. The knowledge retrieval module is used to determine the text similarity, vector similarity, and feedback confidence between the current query request and each knowledge segment based on the user's query request, and outputs the knowledge segment retrieval results based on the text similarity, vector similarity, and feedback confidence of each knowledge segment. The user feedback module is used to collect user feedback information on the knowledge segment retrieval results and natural language answers output for the current query request, and update the feedback confidence of the corresponding knowledge segment and the current query request based on the feedback information, thereby improving retrieval accuracy and answer accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an enterprise knowledge question-answering system, method, device, and medium. Background Technology

[0002] For companies with a long history and deep data accumulation, the data they have built up over time across multiple stages such as R&D, production, and sales, while of significant value, is often hampered by fragmented storage, outdated updates, and a lack of effective retrieval methods. This makes it difficult to quickly access and efficiently utilize the knowledge, thus hindering decision-making efficiency and business development. To address this issue, an enterprise knowledge search platform can be built based on artificial intelligence (AI) big data models and retrieval-enhanced generation (RAG) technology. This platform can meet the core needs of enterprises for intelligent knowledge management and improve overall operational efficiency.

[0003] In current enterprise knowledge management practices, the hybrid retrieval method commonly used in traditional RAG architecture has three limitations: First, it relies on fixed weights and general semantic models, making it difficult to accurately understand internal enterprise terminology and specific contexts, leading to semantic matching biases; second, the retrieval is based solely on static content during knowledge base construction, failing to adapt to the differences in expression between spoken questions and written knowledge descriptions; and finally, the system lacks effective utilization of user interaction feedback, making it difficult for the system to continuously learn and optimize from actual usage scenarios. Summary of the Invention

[0004] Therefore, it is necessary to provide an enterprise knowledge question-and-answer system, method, device, and medium to address the aforementioned technical issues.

[0005] In a first aspect, embodiments of this application provide an enterprise knowledge question-and-answer system, the system comprising:

[0006] The knowledge base module is used to store multiple knowledge fragments of an enterprise;

[0007] The knowledge retrieval module is used to determine the text similarity, vector similarity, and feedback confidence between the current query request and each knowledge fragment based on the user's current query request, and to output the knowledge fragment retrieval results based on the text similarity, vector similarity, and feedback confidence of each knowledge fragment; wherein, the feedback confidence is determined based on the user's feedback information on the knowledge fragment retrieval results output for each query request and the corresponding natural language answers;

[0008] The answer generation module is used to generate a natural language answer to the query request based on the query request and the corresponding knowledge fragment retrieval results, using a large language model.

[0009] The user feedback module is used to collect user feedback information on the knowledge fragment retrieval results and natural language answers output for this query request, and to update the feedback confidence of the corresponding knowledge fragment and the query request based on the feedback information.

[0010] In one embodiment, the knowledge base module is specifically used to: segment the enterprise's knowledge documents to obtain multiple knowledge fragments; add annotation information to each knowledge fragment; convert each knowledge fragment into a numerical vector; and store each knowledge fragment and its corresponding numerical vector in a vector database.

[0011] In one embodiment, updating the confidence level of the corresponding knowledge fragment and the current query request based on the feedback information includes:

[0012] Based on the feedback information of the knowledge fragment retrieval results, the feedback confidence of the corresponding knowledge fragment and the current query request is increased or decreased according to a first proportional coefficient.

[0013] Based on the feedback information of the natural language response, the confidence level of the corresponding knowledge segment and the current query request is increased or decreased according to a second proportional coefficient; the first proportional coefficient is greater than the second proportional coefficient.

[0014] In one embodiment, the feedback information includes implicit feedback information and explicit feedback information. The implicit feedback information includes whether to click to browse or download a document containing the knowledge fragment retrieval results and whether to click to copy the natural language answer. The explicit feedback information includes liking or disliking the knowledge fragment retrieval results and the natural language answer.

[0015] In one embodiment, updating the feedback confidence of each knowledge fragment based on the feedback information includes:

[0016] Based on the display feedback information of the knowledge fragment retrieval results, the feedback confidence of the corresponding knowledge fragment and the current query request is increased or decreased according to the third proportional coefficient.

[0017] Based on the implicit feedback information of the knowledge fragment retrieval results, the feedback confidence of the corresponding knowledge fragment and the current query request is increased or decreased according to the fourth proportional coefficient; the third proportional coefficient is greater than the fourth proportional coefficient.

[0018] Based on the displayed feedback information of the natural language answer, the confidence level of the corresponding knowledge segment and the current query request is increased or decreased according to the fifth proportional coefficient.

[0019] Based on the implicit feedback information of the natural language response, the feedback confidence of the corresponding knowledge segment and the current query request is increased or decreased according to the sixth proportional coefficient; the fifth proportional coefficient is greater than the sixth proportional coefficient.

[0020] In one embodiment, the user feedback module is further configured to: store multiple "query-knowledge fragment" pairs based on the knowledge fragment retrieval results output by the user's previous query requests and the feedback information of the generated natural language answers;

[0021] The knowledge retrieval module is specifically used to match the current query request with the query requests in each of the "query-knowledge fragment" pairs. If the match is successful, the corresponding knowledge fragment retrieval result is directly output based on the matching result. If the match is unsuccessful, the text similarity, vector similarity, and feedback confidence of the current query request and each of the knowledge fragments are determined, and the knowledge fragment retrieval result is output based on the text similarity, vector similarity, and feedback confidence of each of the knowledge fragments.

[0022] In one embodiment, the knowledge retrieval module is further configured to: perform a weighted summation of the text similarity, vector similarity, and feedback confidence of each knowledge fragment to obtain the total matching degree of each knowledge fragment, and output the retrieval results of the top K matching knowledge fragments based on the total matching degree.

[0023] Secondly, embodiments of this application also provide an enterprise knowledge question-answering method, applied to the system described in the first aspect above, the method comprising:

[0024] Based on the user's query request, the text similarity, vector similarity, and feedback confidence of the query request with each of the knowledge fragments are determined, and the knowledge fragment retrieval results are output based on the text similarity, vector similarity, and feedback confidence of each of the knowledge fragments; wherein, the feedback confidence is determined based on the user's feedback information on the knowledge fragment retrieval results output for each query request and the corresponding natural language answers;

[0025] Based on the query request and the corresponding knowledge fragment retrieval results, a natural language answer to the query request is generated using a large language model.

[0026] Collect user feedback information on the knowledge fragment retrieval results and natural language answers output for this query request, and update the feedback confidence of the corresponding knowledge fragment and the query request based on the feedback information.

[0027] Thirdly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the second aspect.

[0028] Fourthly, embodiments of this application also provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in the second aspect.

[0029] The aforementioned enterprise knowledge-based question-answering system, methods, equipment, and media, by introducing dynamic "feedback confidence" and integrating it with semantic and keyword retrieval, enable the retrieval ranking mechanism to not only consider content relevance but also incorporate user wisdom and practical application scenario feedback. The system can learn enterprise-specific terminology, context, and expression habits from real user feedback, significantly improving the accuracy of retrieval results and fundamentally enhancing its ability to match colloquial questions with written documents. Simultaneously, the constructed continuous learning closed-loop mechanism based on user feedback ensures that the value judgment of the knowledge base is no longer static but dynamically evolves with use. This effectively solves the problem of low utilization of feedback data in traditional systems, endowing the system with self-evolution capabilities. The system can automatically optimize without costly retraining, gradually adapting to the enterprise's unique context and user habits, achieving the long-term optimization goal of becoming smarter with use, and continuously improving retrieval accuracy.

[0030] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0031] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0032] Figure 1 This is a structural block diagram of an enterprise knowledge question-and-answer system in one embodiment;

[0033] Figure 2 This is a flowchart illustrating an enterprise knowledge question-and-answer method in one embodiment;

[0034] Figure 3 This is a flowchart illustrating the enterprise knowledge question-and-answer method in another embodiment;

[0035] Figure 4 This is a schematic diagram of the structure of a computer device in one embodiment. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.

[0037] In recent years, artificial intelligence (AI) technologies, represented by Large Language Models (LLMs), have made groundbreaking progress. To overcome the inherent limitations of large models, such as the illusion problem, lagging knowledge updates, and inability to access private data, Retrieval Augmentation (RAG) frameworks have become the mainstream solution in the industry. This framework equips the LLM with an external, updatable knowledge base, enabling it to retrieve the most relevant knowledge fragments to the user's query as a reference before generating an answer, thereby significantly improving the accuracy and credibility of the response.

[0038] In the RAG framework, the performance of the retrieval module is the core bottleneck determining the overall system efficiency. To balance recall and precision, hybrid retrieval strategies are widely adopted. Traditional hybrid retrieval typically combines two classic algorithms: one is sparse retrieval based on word frequency (text similarity), which focuses on the lexical matching between the query and the document, excelling at capturing precise keyword overlap; the other is dense vector retrieval based on deep learning (vector similarity), which maps text to a high-dimensional vector space and captures semantic relevance by calculating vector distance, enabling it to understand synonyms and concept associations. Although traditional hybrid retrieval combines the advantages of the above two methods, its retrieval performance is still unsatisfactory in complex enterprise-level knowledge scenarios. This is mainly due to the following underlying reasons: First, the semantics of internal enterprise terms, industry jargon, project codes, etc., may not be fully understood by general models, leading to biases in vector retrieval; second, there is a difference between the colloquial expressions of user queries and the rigorous written descriptions in the knowledge base; third, once the knowledge base is built, its representation is basically fixed. They cannot dynamically adjust and optimize based on real-world user interaction data, and lack the ability to learn from real feedback. Furthermore, traditional hybrid retrieval systems increasingly suffer from retrieval latency issues when dealing with massive enterprise-level knowledge bases. Each retrieval request requires simultaneous text matching and semantic vector calculation, resulting in high computational demands and response times that struggle to meet the needs of high-frequency, real-time interactive enterprise applications.

[0039] Essentially, existing hybrid retrieval algorithms are a "static" matching process that relies solely on the document content itself, completely ignoring the valuable implicit feedback generated by users during system usage. For example, when a user ultimately adopts a search result as decision support, or spends a considerable amount of time browsing a particular passage, these actions themselves are strong evidence that the passage is highly relevant to the query. However, in existing technologies, this valuable feedback data is wasted and not used to continuously optimize the retrieval model, preventing the system from evolving to adapt to the unique context and preferences of specific businesses or users.

[0040] Based on the above problems, embodiments of this application provide an enterprise knowledge question-and-answer system, such as... Figure 1 As shown, the system includes a knowledge base module 10, a knowledge retrieval module 20, an answer generation module 30, and a user feedback module 40.

[0041] The knowledge base module 10 stores multiple knowledge fragments of the enterprise. Specifically, it systematically processes the enterprise's original knowledge documents to make them suitable for subsequent retrieval and application. First, long knowledge documents are segmented according to logical or semantic units, and annotation information (such as source, type, importance, etc.) is added to each fragment to form multiple knowledge fragments. Then, a pre-trained embedding model is used to convert the knowledge fragments into numerical vectors, and the embedded vectors and their corresponding original knowledge fragments are stored in a vector database to build an efficient searchable knowledge base that supports fast querying and matching based on semantics and text.

[0042] The knowledge retrieval module 20 is used to determine the text similarity, vector similarity, and feedback confidence between the current query request and each knowledge fragment based on the user's current query request, and to output the knowledge fragment retrieval results based on the text similarity, vector similarity, and feedback confidence of each knowledge fragment; wherein, the feedback confidence is determined based on the user's feedback information on the knowledge fragment retrieval results output for each query request and the corresponding natural language answers.

[0043] Specifically, prior to the user's current query request, the system continuously collects and analyzes the user's previous query requests, as well as the user's feedback information on the knowledge fragment retrieval results and corresponding natural language answers output by the previous query requests. Based on the feedback information, the system continuously updates and optimizes the feedback confidence of the relevant knowledge fragments of previous query requests and retrieval outputs, and stores it in the feedback confidence database.

[0044] The knowledge retrieval module 20 in this embodiment is built on the RAG framework to respond to the user's query request. Based on the query request, it calculates the text similarity and vector similarity between the query request and each knowledge segment, and determines the feedback confidence of the query request and each knowledge segment based on the feedback confidence data stored in the feedback confidence database. Specifically, the knowledge retrieval module simultaneously calculates the vector similarity between the query request and each knowledge segment in the vector semantic space, the text similarity in keyword matching, and matches the current query request with previous query requests to identify queries that are repeated or highly similar to the current query request. Based on the feedback confidence of the knowledge segments corresponding to these historical query requests that are repeated or highly similar to the current query request, it determines the feedback confidence of the current query request and each knowledge segment. The knowledge retrieval module uses a weighted model to fuse the text similarity, vector similarity, and feedback confidence of each knowledge segment, filters and sorts the knowledge segments, and selects relevant knowledge segments that both fit the original meaning of the question and conform to the actual value judgment of the user group.

[0045] Compared to the learning-based ranking commonly used in current mainstream RAG frameworks, this application saves significant maintenance costs previously associated with manual annotation and model iteration. It replaces expensive manual annotation with lightweight user feedback, enabling data-driven real-time optimization. Feedback confidence is updated in real time, directly impacting the next retrieval. Compared to learning-based ranking methods that require weeks of data collection and model training, this system's optimization is immediate and agile.

[0046] The answer generation module 30 is used to generate a natural language answer to the query request based on the query request and the corresponding knowledge fragment retrieval results, using a large language model.

[0047] Specifically, the answer generation module combines relevant knowledge fragments provided by the knowledge retrieval module with the user's query request, and uses a Large Language Model (LLM) to generate a natural language answer. The answer generation module constructs a structured prompt based on preset prompt word specifications. This prompt typically includes system role instructions, user questions, relevant knowledge fragments, historical dialogues, and output format requirements. Leveraging its powerful semantic understanding and content generation capabilities, the Large Language Model performs deep reasoning and integration on the user questions and relevant knowledge fragments in the prompt, generating a direct, coherent answer that aligns with the question's intent.

[0048] The user feedback module 40 is used to collect user feedback information on the knowledge fragment retrieval results and natural language answers output by the user for this query request, and to update the feedback confidence of the corresponding knowledge fragment and the query request based on the feedback information.

[0049] Specifically, the user feedback module collects user feedback information regarding the knowledge fragment retrieval results and natural language responses for this query request, and continuously updates the feedback confidence level of the corresponding knowledge fragment and the current query request based on this feedback information. It is understood that when there are historical query requests that are identical or highly similar to the current query request, the redundant query requests are deleted, and the feedback confidence level of the corresponding knowledge fragment and the query request is updated. If no such historical query requests exist, the feedback confidence level for the current query request is directly generated. When the user queries the same request again in the future, the system will update and optimize the current feedback confidence level based on the information from future query requests.

[0050] The enterprise knowledge question-answering system in this application introduces a dynamic "feedback confidence level" and integrates it with semantic and keyword retrieval. This allows the retrieval ranking mechanism to consider not only content relevance but also user wisdom and practical application scenario feedback. The system can learn enterprise-specific terminology, context, and expression habits from real user feedback, significantly improving the accuracy of retrieval results. The system's ability to match colloquial questions with written documents is fundamentally improved, thereby enhancing the accuracy of answers. Simultaneously, the constructed continuous learning closed-loop mechanism based on user feedback ensures that the value judgment of the knowledge base is no longer static but dynamically evolves with use. This effectively solves the problem of low utilization of feedback data in traditional systems, endowing the system with self-evolution capabilities. The system can automatically optimize without costly retraining, gradually adapting to the enterprise's unique context and user habits, achieving the long-term optimization goal of becoming smarter with use, continuously improving retrieval accuracy, and enhancing the accuracy of question answers.

[0051] In one embodiment, the knowledge base module is specifically used to: segment the enterprise's knowledge documents to obtain multiple knowledge fragments; add annotation information to each knowledge fragment; convert each knowledge fragment into a numerical vector; and store each knowledge fragment and its corresponding numerical vector in a vector database.

[0052] The feedback information is determined by user behavior data regarding the search results for knowledge fragments and their responses to natural language answers. The system automatically collects user behavior data related to the retrieved knowledge fragments and generated natural language answers, such as clicking to browse, download, copy, like, or dislike. Specifically, the feedback information includes implicit and explicit feedback. Implicit feedback includes whether the user clicks to browse or download a document containing the search results for the knowledge fragment and whether they click to copy the natural language answer. Explicit feedback includes liking or disliking the search results for the knowledge fragment and the natural language answer.

[0053] Specifically, for a query request, the knowledge retrieval module outputs the top K knowledge fragments as search results. The implicit feedback includes whether to click to browse or download a document containing these knowledge fragments. In other embodiments, the answer generation module constructs a structured prompt based on preset prompt word specifications. The system inputs the query request and the top K knowledge fragment search results into the prompt. In this case, the user may choose not to use the knowledge fragment search results; that is, the user actively deletes one or more knowledge fragment search results. This user behavior can be understood as implicit negative feedback, which correspondingly reduces the confidence level of the corresponding knowledge fragment in relation to the same or similar query request. Knowledge fragment search results adopted by the user are not deleted. This user behavior can be understood as implicit positive feedback, which correspondingly increases the confidence level of the corresponding knowledge fragment in relation to the same or similar query request. Simultaneously, the implicit feedback information also includes whether the user clicked to copy the natural language answer. If the user clicked to copy the natural language answer, it is implicit positive feedback, indicating that the natural language answer is highly relevant to the query request; if the user did not click to copy the natural language answer, it is implicit negative feedback, indicating that the natural language answer is not very relevant to the query request or is irrelevant. In other embodiments, the implicit feedback information may also include the duration of the user's browsing of the knowledge fragment search results and the natural language answer. For example, when the user browses the natural language answer for a long time, it can be understood as implicit positive feedback information.

[0054] In one embodiment, updating the feedback confidence of the corresponding knowledge fragment and the current query request based on the feedback information involves: increasing or decreasing the feedback confidence of the corresponding knowledge fragment and the current query request according to a first proportional coefficient based on the feedback information of the knowledge fragment retrieval results; and increasing or decreasing the feedback confidence of the corresponding knowledge fragment and the current query request according to a second proportional coefficient based on the feedback information of the natural language response. The first proportional coefficient is greater than the second proportional coefficient, meaning that the weight of the user's feedback information on the knowledge fragment retrieval results is greater than the weight of the user's feedback information on the natural language response.

[0055] The feedback information in this embodiment includes user feedback on each relevant knowledge fragment in the knowledge fragment retrieval results, as well as user feedback on the generated natural language answer. The system can determine the corresponding relevant knowledge fragment in the answer generation module based on the generated natural language answer.

[0056] For example, when a user likes or dislikes an answer, it slightly increases or decreases the confidence level of all related knowledge fragments in response to the current user's question and to similar or identical previous queries. Conversely, liking or disliking a single knowledge fragment significantly increases or decreases its confidence level in response to the current user's question and to similar or identical previous queries. The system updates the feedback data corresponding to the current user's question in the feedback confidence database. During this update process, the system can merge the current query request with similar or identical requests from previous queries. If no similar or identical historical query request exists, a new entry is created in the feedback confidence database. The system will then update and optimize based on information from future queries.

[0057] In one embodiment, updating the feedback confidence of each knowledge fragment based on the feedback information includes: increasing or decreasing the feedback confidence of the corresponding knowledge fragment and the current query request according to a third proportional coefficient based on the displayed feedback information of the knowledge fragment retrieval results; increasing or decreasing the feedback confidence of the corresponding knowledge fragment and the current query request according to a fourth proportional coefficient based on the implicit feedback information of the knowledge fragment retrieval results; the third proportional coefficient is greater than the fourth proportional coefficient; that is, the user's weight for the displayed feedback information of the knowledge fragment retrieval results is greater than the user's weight for the implicit feedback information of the knowledge fragment retrieval results.

[0058] Based on the explicit feedback information of the natural language response, the confidence level of the corresponding knowledge segment and the current query request is increased or decreased according to a fifth proportional coefficient; based on the implicit feedback information of the natural language response, the confidence level of the corresponding knowledge segment and the current query request is increased or decreased according to a sixth proportional coefficient; the fifth proportional coefficient is greater than the sixth proportional coefficient. That is, the weight of the user's explicit feedback information to the natural language response is greater than the weight of the user's implicit feedback information to the natural language response.

[0059] In one embodiment, the user feedback module is further configured to: store multiple "query-knowledge fragment" pairs based on the knowledge fragment retrieval results output by the user's previous query requests and the feedback information of the generated natural language answers; the knowledge retrieval module is specifically configured to match the current query request with the query requests in each of the "query-knowledge fragment" pairs; if the match is successful, the corresponding knowledge fragment retrieval result is directly output based on the matching result; if the match is unsuccessful, the text similarity, vector similarity, and feedback confidence of the current query request and each of the knowledge fragments are determined, and the knowledge fragment retrieval result is output based on the text similarity, vector similarity, and feedback confidence of each of the knowledge fragments.

[0060] In this embodiment, the user feedback module, based on historical feedback records, establishes a cache index for certain queries and their corresponding high-confidence knowledge fragments after multiple verifications. When encountering similar or highly relevant queries subsequently, the system can directly return these proven knowledge fragment results, significantly reducing redundant calculations and achieving a substantial improvement in retrieval speed. This effectively solves the response latency bottleneck in enterprise applications. Under actual load, for common cache hits (accounting for approximately 40% of daily queries), the average system response time has been drastically reduced from 850 milliseconds to less than 25 milliseconds. This not only greatly improves the user experience but also effectively reduces backend computing resource consumption, enabling the system to more easily handle the challenges of massive enterprise-level knowledge bases and high-concurrency queries.

[0061] In one embodiment, the knowledge retrieval module is further configured to: perform a weighted summation of the text similarity, vector similarity, and feedback confidence of each knowledge fragment to obtain the total matching degree of each knowledge fragment, and output the retrieval results of the top K matching knowledge fragments based on the total matching degree.

[0062] This application also provides an enterprise knowledge question-answering method, applicable to systems as described in any of the above embodiments, such as... Figure 2 As shown, the method includes the following steps:

[0063] Step 201: Based on the user's current query request, determine the text similarity, vector similarity, and feedback confidence between the current query request and each of the knowledge fragments, and output the knowledge fragment retrieval results based on the text similarity, vector similarity, and feedback confidence of each of the knowledge fragments; wherein, the feedback confidence is determined based on the user's feedback information on the knowledge fragment retrieval results output for each query request and the corresponding natural language answers;

[0064] Step 202: Based on the current query request and the corresponding knowledge fragment retrieval results, a natural language answer to the current query request is generated using a large language model;

[0065] Step 203: Collect the user's feedback information on the knowledge fragment retrieval results and natural language answers output for this query request, and update the feedback confidence of the corresponding knowledge fragment and the query request based on the feedback information.

[0066] Figure 3 This is a flowchart illustrating one embodiment of an enterprise knowledge question-answering method, which includes the following steps:

[0067] Step S301: The original knowledge document of the enterprise is segmented and labeled to obtain multiple knowledge blocks (i.e. knowledge fragments).

[0068] Step S302: Use a pre-trained embedding model to convert knowledge blocks into numerical vectors, and store the embedded vectors and their corresponding original text fragments in a vector database.

[0069] Step S303: Based on the vector database and the feedback confidence database, perform a mixed retrieval of the user's question to obtain relevant knowledge fragments.

[0070] Step S304: Based on the user's question and related knowledge fragments, a large language model is used to generate a natural language answer.

[0071] Step S305: Collect user feedback information on relevant knowledge fragments and answers, and update the feedback confidence of relevant knowledge fragments based on the feedback information.

[0072] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0073] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0074] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above embodiments of the enterprise knowledge question-answering method.

[0075] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0076] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0077] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An enterprise knowledge question answering system, characterized by, The system comprises: a knowledge base module configured to store a plurality of knowledge segments of an enterprise; a knowledge retrieval module configured to determine, based on a current query request of a user, a text similarity, a vector similarity and a feedback confidence of the current query request and each of the knowledge segments, and output a knowledge segment retrieval result based on the text similarity, the vector similarity and the feedback confidence of each of the knowledge segments; wherein the feedback confidence is determined based on feedback information of the user on a knowledge segment retrieval result and a corresponding natural language answer output for a previous query request; a response generation module configured to generate, based on the current query request and the corresponding knowledge segment retrieval result, a natural language answer to the current query request by using a large language model; a user feedback module configured to collect feedback information of the user on the knowledge segment retrieval result and the natural language answer output for the current query request, and update the feedback confidence of the corresponding knowledge segment and the current query request based on the feedback information.

2. The system of claim 1, wherein, The knowledge base module is specifically configured to: split each knowledge document of the enterprise to obtain a plurality of knowledge segments; add label information to each of the knowledge segments, convert each of the knowledge segments into a numerical vector, and store each of the knowledge segments and the corresponding numerical vector in a vector database.

3. The system of claim 1, wherein, The updating of the feedback confidence of the corresponding knowledge segment and the current query request based on the feedback information comprises: increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a first proportional coefficient based on the feedback information of the knowledge segment retrieval result; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a second proportional coefficient based on the feedback information of the natural language answer; the first proportional coefficient is greater than the second proportional coefficient.

4. The system of claim 1, wherein, The feedback information comprises implicit feedback information and explicit feedback information, the implicit feedback information comprises whether to click to browse or download a document containing the knowledge segment retrieval result and whether to click to copy the natural language answer, and the explicit feedback information comprises liking or disliking the knowledge segment retrieval result and the natural language answer.

5. The system of claim 4, wherein, The updating of the feedback confidence of the corresponding knowledge segment and the current query request based on the feedback information comprises: increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a third proportional coefficient based on the explicit feedback information of the knowledge segment retrieval result; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a fourth proportional coefficient based on the implicit feedback information of the knowledge segment retrieval result; the third proportional coefficient is greater than the fourth proportional coefficient; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a fifth proportional coefficient based on the explicit feedback information of the natural language answer; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a sixth proportional coefficient based on the implicit feedback information of the natural language answer; the fifth proportional coefficient is greater than the sixth proportional coefficient.

6. The system of claim 1, wherein, The user feedback module is further configured to: retrieve results of knowledge segments based on the feedback information of the user on the previous query request and the generated natural language answer; The knowledge retrieval module is specifically configured to match the current query request with the query request in each of the "query-knowledge segment" pairs, and if the matching is successful, directly output the corresponding knowledge segment retrieval result based on the matching result; If the matching is not successful, determine the text similarity, vector similarity and feedback confidence of the current query request and each of the knowledge segments, and output the knowledge segment retrieval result based on the text similarity, vector similarity and feedback confidence of each of the knowledge segments.

7. The system of claim 1, wherein, The knowledge retrieval module is further configured to: weight and sum the text similarity, vector similarity and feedback confidence of each of the knowledge segments to obtain the total matching degree of each of the knowledge segments, and output the top K matching knowledge segment retrieval results based on the total matching degree.

8. An enterprise knowledge question answering method, characterized by, The method is applied to the system of any one of claims 1-7, and the method comprises: based on the current query request of the user, determine the text similarity, vector similarity and feedback confidence of the current query request and each of the knowledge segments, and output the knowledge segment retrieval result based on the text similarity, vector similarity and feedback confidence of each of the knowledge segments; wherein the feedback confidence is determined based on the feedback information of the user on the previous query request and the corresponding natural language answer; based on the current query request and the corresponding knowledge segment retrieval result, generate a natural language answer to the current query request using a large language model; collect the feedback information of the user on the knowledge segment retrieval result and the natural language answer of the current query request, and update the feedback confidence of the corresponding knowledge segment and the current query request based on the feedback information.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of claim 8. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-9. The processor executes the computer program to implement the method of claim 8.

Images