Enterprise knowledge question-answering system, method, equipment and medium

Abstract

The invention relates to an enterprise knowledge question-answering system, method and device and a medium, and the enterprise knowledge question-answering system comprises a knowledge base module, a knowledge retrieval module, an answer generation module and a user feedback module. Determining the text similarity, the vector similarity and the feedback confidence of the query request and each knowledge fragment, and outputting a knowledge fragment retrieval result based on the text similarity, the vector similarity and the feedback confidence of each knowledge fragment; and the user feedback module is used for collecting the knowledge fragment retrieval result output by the user for the query request and the feedback information of the natural language answer, and updating the feedback confidence of the corresponding knowledge fragment and the query request based on the feedback information, so that the retrieval precision and the answer accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an enterprise knowledge question-answering system, method, device and medium. BACKGROUND

[0002] For enterprises with a long history and deep data accumulation, the multi-link data of research and development, production, sales and the like formed over a long period of time has important value, but due to scattered storage, lagging updates and lack of effective retrieval means, knowledge is difficult to be quickly acquired and efficiently utilized, which restricts decision-making efficiency and business development. To solve this problem, an enterprise knowledge search platform can be constructed based on artificial intelligence (AI) large models and retrieval augmented generation (RAG) technology to meet the core needs of intelligent knowledge management of enterprises and improve overall operational efficiency.

[0003] In current enterprise knowledge management practices, the hybrid retrieval method commonly used in traditional RAG architecture has three limitations: first, it relies on fixed weights and general semantic models, which makes it difficult to accurately understand internal professional terms and specific contexts, resulting in semantic matching bias; second, retrieval is based only on static content at the time of knowledge base construction, which cannot adapt to the expression differences between spoken questions and written knowledge expressions; and third, the system lacks effective use of user interaction feedback, making it difficult for the system to continuously learn and optimize from actual use scenarios. SUMMARY

[0004] Therefore, it is necessary to provide an enterprise knowledge question-answering system, method, device and medium to solve the above technical problems.

[0005] In a first aspect, an enterprise knowledge question-answering system is provided, which includes:

[0006] a knowledge base module configured to store a plurality of knowledge segments of an enterprise;

[0007] a knowledge retrieval module configured to determine text similarity, vector similarity and feedback confidence between a current query request of a user and each of the knowledge segments based on the current query request, and output a knowledge segment retrieval result based on the text similarity, vector similarity and feedback confidence of each of the knowledge segments; wherein the feedback confidence is determined based on feedback information of the user on the knowledge segment retrieval result and the corresponding natural language answer output for previous query requests;

[0008] a response generation module configured to generate a natural language answer to the current query request using a large language model based on the current query request and the corresponding knowledge segment retrieval result;

[0009] a user feedback module configured to collect feedback information of a user on the knowledge snippet retrieval result and the natural language answer output for the query request, and update a feedback confidence of the corresponding knowledge snippet and the query request based on the feedback information.

[0010] In one of the embodiments, the knowledge base module is specifically configured to: split each knowledge document of the enterprise to obtain a plurality of knowledge snippets; add label information to each of the knowledge snippets, convert each of the knowledge snippets into a numerical vector, and store each of the knowledge snippets and the corresponding numerical vector in a vector database.

[0011] In one of the embodiments, the updating of the feedback confidence of the corresponding knowledge snippet and the query request based on the feedback information comprises:

[0012] increasing or decreasing the feedback confidence of the corresponding knowledge snippet and the query request based on the feedback information of the knowledge snippet retrieval result according to a first proportional coefficient;

[0013] increasing or decreasing the feedback confidence of the corresponding knowledge snippet and the query request based on the feedback information of the natural language answer according to a second proportional coefficient; the first proportional coefficient is greater than the second proportional coefficient.

[0014] In one of the embodiments, the feedback information comprises implicit feedback information and explicit feedback information, the implicit feedback information comprises whether to click to browse or download a document containing the knowledge snippet retrieval result and whether to click to copy the natural language answer, and the explicit feedback information comprises liking or disliking the knowledge snippet retrieval result and the natural language answer.

[0015] In one of the embodiments, the updating of the feedback confidence of each of the knowledge snippets based on the feedback information comprises:

[0016] increasing or decreasing the feedback confidence of the corresponding knowledge snippet and the query request based on the explicit feedback information of the knowledge snippet retrieval result according to a third proportional coefficient;

[0017] increasing or decreasing the feedback confidence of the corresponding knowledge snippet and the query request based on the implicit feedback information of the knowledge snippet retrieval result according to a fourth proportional coefficient; the third proportional coefficient is greater than the fourth proportional coefficient;

[0018] increasing or decreasing the feedback confidence of the corresponding knowledge snippet and the query request based on the explicit feedback information of the natural language answer according to a fifth proportional coefficient;

[0019] increase or decrease the feedback confidence of the corresponding knowledge segment and the query request this time based on the implicit feedback information of the natural language answer; the fifth proportional coefficient is greater than the sixth proportional coefficient.

[0020] In one of the embodiments, the user feedback module is further configured to: based on the feedback information of the knowledge segment retrieval result and the generated natural language answer output by the user for the previous query request, store a plurality of "query-knowledge segment" pairs;

[0021] The knowledge retrieval module is specifically configured to: match the query request this time with the query request in each of the "query-knowledge segment" pairs, if the matching is successful, directly output the corresponding knowledge segment retrieval result based on the matching result; if the matching is not successful, determine the text similarity, vector similarity and feedback confidence of the query request this time and each of the knowledge segments, and output the knowledge segment retrieval result based on the text similarity, vector similarity and feedback confidence of each of the knowledge segments.

[0022] In one of the embodiments, the knowledge retrieval module is further configured to: weight and sum the text similarity, vector similarity and feedback confidence of each of the knowledge segments to obtain the total matching degree of each of the knowledge segments, and output the top K matching knowledge segment retrieval results based on the total matching degrees.

[0023] In a second aspect, the embodiments of the present application further provide an enterprise knowledge question and answer method, applied to the system as described in the first aspect, and the method comprises:

[0024] Based on the query request this time of the user, determine the text similarity, vector similarity and feedback confidence of the query request this time and each of the knowledge segments, and output the knowledge segment retrieval result based on the text similarity, vector similarity and feedback confidence of each of the knowledge segments; wherein the feedback confidence is determined based on the feedback information of the knowledge segment retrieval result and the corresponding natural language answer output by the user for the previous query request;

[0025] Based on the query request this time and the corresponding knowledge segment retrieval result, generate a natural language answer for the query request this time using a large language model;

[0026] Collect the feedback information of the knowledge segment retrieval result and the natural language answer output by the user for the query request this time, and update the feedback confidence of the corresponding knowledge segment and the query request this time based on the feedback information.

[0027] In a third aspect, the embodiments of the present application further provide a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method as described in the second aspect.

[0028] In a fourth aspect, the embodiments of the present application further provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to realize the method of the second aspect.

[0029] The enterprise knowledge Q&A system, method, device and medium described above introduce dynamic feedback confidence and combine with semantic and keyword retrieval, so that the retrieval ranking mechanism not only considers content relevance, but also integrates user wisdom and actual application scenario feedback. The system can learn enterprise-specific terminology, context and expression habits from real user feedback, and the accuracy of the retrieval result is significantly improved. The system's matching ability for oral questions and written documents is fundamentally improved. At the same time, the continuous learning closed-loop mechanism based on user feedback is constructed, so that the value judgment of the knowledge base is no longer static, but dynamically evolves with use. The problem of low utilization rate of feedback data in traditional systems is effectively solved, and the system is given the ability of self-evolution, which can be automatically optimized without expensive retraining. It can gradually adapt to the specific context and user habits of the enterprise, achieve the long-term optimization goal of becoming more intelligent with use, and continuously improve the retrieval precision.

[0030] Details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0031] The drawings described herein are intended to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0032] Figure 1 is a structural block diagram of an enterprise knowledge Q&A system in an embodiment;

[0033] Figure 2 is a flowchart of an enterprise knowledge Q&A method in an embodiment;

[0034] Figure 3 is a flowchart of an enterprise knowledge Q&A method in another embodiment;

[0035] Figure 4 is a structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present application more apparent, the present application is described and explained below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0037] In recent years, artificial intelligence (AI) technologies represented by large language models (LLMs) have made breakthrough progress. In order to overcome the inherent hallucination problem of large models, the lag of knowledge updating, and the limitation of not being able to access private data, the retrieval-augmented generation (RAG) framework has become the mainstream solution in the industry. By providing an external and updatable knowledge base for LLMs, the framework enables them to retrieve the most relevant knowledge fragments to user queries as reference before generating answers, thereby significantly improving the accuracy and credibility of answers.

[0038] In the RAG framework, the performance of the retrieval module is the core bottleneck that determines the overall system efficiency. In order to balance the recall and accuracy of retrieval, hybrid retrieval strategies are widely used. Traditional hybrid retrieval usually combines two classic algorithms: one is sparse retrieval based on word frequency (text similarity), which focuses on the degree of lexical matching between queries and documents, and is good at capturing exact keyword overlaps; the other is dense vector retrieval based on deep learning (vector similarity), which maps text to high-dimensional vector space and captures semantic relevance by calculating vector distance, capable of understanding synonyms and concept associations. Although traditional hybrid retrieval combines the advantages of the above two methods, its retrieval effect is still not satisfactory in complex enterprise-level knowledge scenarios. This is mainly due to the following deep-seated reasons: reason one is that the semantics of enterprise-specific terms, industry jargon, project codes, etc. may not be fully understood by general models, leading to deviations in vector retrieval; reason two is the difference between colloquial expressions in user queries and formal written descriptions in the knowledge base; reason three is that once the knowledge base is constructed, its representation is basically fixed. They cannot be dynamically adjusted and optimized according to user interaction data in actual use, lacking the ability to learn from real feedback. In addition, traditional hybrid retrieval has increasingly highlighted the problem of retrieval delay when facing massive enterprise-level knowledge bases. Each retrieval request needs to perform both text matching and semantic vector calculation, which is computationally intensive and difficult to meet the response speed requirements of high-frequency, real-time interaction in enterprise applications.

[0039] Essentially, existing hybrid retrieval algorithms are a "static" matching process that only relies on document content itself, completely ignoring valuable implicit feedback information generated by users during system use. For example, when a user ultimately adopts a certain retrieval result as decision support, or spends a long time browsing a fragment, these behaviors themselves are strong evidence that "the fragment is highly relevant to the query". However, in existing technologies, these valuable feedback data are wasted and cannot be used to continuously optimize the retrieval model, resulting in the system being unable to evolve itself to adapt to the unique context and preferences of a particular enterprise or user.

[0040] Based on the above problems, an enterprise knowledge Q&A system is provided, as follows:Figure 1 As shown, the system comprises a knowledge base module 10, a knowledge retrieval module 20, an answer generation module 30, and a user feedback module 40.

[0041] The knowledge base module 10 is used to store multiple knowledge segments of the enterprise. Specifically, it is used to systematically process the original knowledge documents of the enterprise, making them suitable for subsequent retrieval and application. First, long knowledge documents are divided into logical or semantic units, and each segment is added with annotation information (such as source, type, importance, etc.), forming multiple knowledge segments. Then, a pre-trained embedding model is used to convert the knowledge segments into numerical vectors, and the embedded vectors and their corresponding original knowledge segments are stored in a vector database to construct an efficient knowledge base that supports fast query and matching based on semantics and text.

[0042] The knowledge retrieval module 20 is used to determine the text similarity, vector similarity, and feedback confidence between the current query request of the user and each knowledge segment based on the current query request of the user, and to output the knowledge segment retrieval result based on the text similarity, vector similarity, and feedback confidence of each knowledge segment. The feedback confidence is determined based on the feedback information of the user on the knowledge segment retrieval results and the corresponding natural language answers of the previous query requests.

[0043] Specifically, before the current query request of the user, the system continuously collects and analyzes the previous query requests of the user and the feedback information of the user on the knowledge segment retrieval results and the corresponding natural language answers of the previous query requests using the user feedback module 40, and continuously updates and optimizes the feedback confidence of the relevant knowledge segments of the previous query requests and retrieval outputs based on the feedback information, and stores them in the feedback confidence library.

[0044] The knowledge retrieval module 20 of the embodiment of the present application is built based on the RAG framework, is used for responding to the query request of the user this time, and calculates the text similarity, the vector similarity of the query request this time and each knowledge fragment based on the query request this time, and simultaneously determines the feedback confidence of the query request this time and each knowledge fragment according to the feedback confidence data stored in the feedback confidence library. Specifically, the knowledge retrieval module synchronously calculates the vector similarity of the query request and each knowledge fragment in the vector semantic space, the text similarity on the keyword matching, and matches the query request this time with the historical query requests, and matches out the repeated or highly similar queries in the historical query requests with the query request this time, and determines the feedback confidence of the query request this time and each knowledge fragment according to the feedback confidence of the knowledge fragments corresponding to the historical query requests which are repeated or highly similar to the query request this time. The knowledge retrieval module fuses the text similarity, the vector similarity and the feedback confidence of each knowledge fragment through a weighting model, filters and sorts the knowledge fragments, and filters out the related knowledge fragments which are consistent with the intention of the problem and meet the actual value judgment of the user group.

[0045] Compared with the learning ranking generally used in the current mainstream RAG framework, the present application saves a large amount of maintenance cost originally used for artificial labeling and model iteration, replaces the expensive artificial labeling with light user feedback, and realizes real-time optimization driven by data. The update of the feedback confidence is completed in real time, which can directly affect the next retrieval. Compared with the learning ranking method which needs weeks of data collection and model training, the optimization of the system is instant and agile.

[0046] The answer generation module 30 is used for generating a natural language answer of the query request this time by using a large language model based on the query request this time and the corresponding knowledge fragment retrieval result.

[0047] Specifically, the answer generation module is responsible for combining the related knowledge fragments provided by the knowledge retrieval module with the query request of the user, and generating a natural language answer by using a large language model (LLM). The answer generation module will construct a structured prompt according to the preset prompt word specification, and the prompt usually includes system role instructions, user problems, related knowledge fragments, historical dialogues, and output format requirements. The large language model performs deep reasoning and integration on the user problems and related knowledge fragments in the prompt based on its powerful semantic understanding and content generation capability, and generates an answer which is direct, coherent and consistent with the intention of the problem.

[0048] The user feedback module 40 is used for collecting the feedback information of the user on the knowledge fragment retrieval result and the natural language answer output by the query request this time, and updating the feedback confidence of the corresponding knowledge fragment and the query request this time based on the feedback information.

[0049] Specifically, the user feedback module collects feedback information of a user on the knowledge fragment retrieval result and the natural language answer output by the query request, and continuously updates the feedback confidence of the corresponding knowledge fragment and the query request according to the feedback information. It can be understood that when there is a historical query request that is the same as or highly similar to the query request in the previous query request, the same redundant query request is deleted, and the feedback confidence of the corresponding knowledge fragment and the query request is updated. If not, the feedback confidence of the query request is directly generated. When the user queries the request again in the future, the system will update and optimize the current feedback confidence according to the feedback information of the future query request.

[0050] The enterprise knowledge Q&A system of the embodiment of the application introduces dynamic "feedback confidence" and integrates it with semantic and keyword retrieval, so that the retrieval and sorting mechanism not only considers content relevance, but also incorporates user wisdom and actual application scenario utility feedback. The system can learn from real user feedback the specific terminology, context and expression habits within the enterprise, significantly improve the accuracy of the retrieval result, and fundamentally improve the matching ability of the system for oral questions and written documents, thereby improving the accuracy of the answers. At the same time, the continuous learning closed-loop mechanism based on user feedback is constructed, so that the value judgment of the knowledge base is no longer static, but dynamically evolves with use. The problem of low utilization rate of feedback data in traditional systems is effectively solved, and the system is given the ability to evolve itself, so that the system can be automatically optimized without expensive retraining. It can gradually adapt to the specific context and user habits of the enterprise, achieve the long-term optimization goal of becoming smarter over time, continuously improve the retrieval accuracy, and improve the accuracy of the answers to the questions.

[0051] In one of the embodiments, the knowledge base module is specifically configured to: divide each knowledge document of the enterprise to obtain a plurality of knowledge fragments; add label information to each knowledge fragment, convert each knowledge fragment into a numerical vector, and store each knowledge fragment and the corresponding numerical vector in a vector database.

[0052] The feedback information is determined by the behavior data of the user on the retrieval result of the knowledge fragment and the natural language answer. The system automatically collects the behavior data of the user on the retrieval output knowledge fragment and the generated natural language answer, such as click browsing, downloading, click copying, liking or disliking, etc. Specifically, the feedback information includes implicit feedback information and explicit feedback information. The implicit feedback information includes whether to click to browse or download the document containing the knowledge fragment retrieval result and whether to click to copy the natural language answer. The explicit feedback information includes liking or disliking the knowledge fragment retrieval result and the natural language answer.

[0053] Specifically, for a query request, the knowledge retrieval module outputs a retrieval result of top K knowledge segments based on the query request. The implicit feedback information includes whether to click to browse or download a document containing the knowledge segments. In other embodiments, the answer generation module constructs a structured prompt according to a preset prompt word specification. The system inputs the query request and the retrieval result of the top K knowledge segments into the prompt. At this time, the user can not use the knowledge segment retrieval result, that is, the user actively deletes one or more knowledge segment retrieval results. This user behavior can also be understood as implicit negative feedback, which will correspondingly reduce the feedback confidence of the corresponding knowledge segment and the same or similar query request. For the knowledge segment retrieval result adopted by the user, no deletion is performed. This user behavior can be understood as implicit positive feedback, which will correspondingly increase the feedback confidence of the corresponding knowledge segment and the same or similar query request. At the same time, the implicit feedback information also includes whether the user clicks to copy the natural language answer. If the user clicks to copy the natural language answer, it is implicit positive feedback, indicating that the natural language answer has high relevance to the query request. If the user does not click to copy the natural language answer, it is implicit negative feedback, indicating that the natural language answer has little or no relevance to the query request. In other embodiments, the implicit feedback information can also include the time length of the user browsing the knowledge segment retrieval result and the natural language answer. For example, when the user browses the natural language answer for a long time, it can be understood as implicit positive feedback information.

[0054] In one embodiment, the feedback information is used to update the feedback confidence of the corresponding knowledge segment and the query request: based on the feedback information of the knowledge segment retrieval result, the feedback confidence of the corresponding knowledge segment and the query request is increased or decreased by a first proportion coefficient; based on the feedback information of the natural language answer, the feedback confidence of the corresponding knowledge segment and the query request is increased or decreased by a second proportion coefficient; the first proportion coefficient is greater than the second proportion coefficient, that is, the weight of the feedback information of the user on the knowledge segment retrieval result is greater than the weight of the feedback information of the user on the natural language answer.

[0055] The feedback information in the embodiments of the present application includes user feedback on each relevant knowledge segment in the knowledge segment retrieval result and user feedback on the generated natural language answer. The system can determine the corresponding input relevant knowledge segment in the answer generation module according to the generated natural language answer.

[0056] For example, when the user likes or dislikes the generated answer, the feedback confidence of all related knowledge segments for the current user question and the same or similar historical query request will be slightly increased or decreased, and the feedback confidence of a single knowledge segment for the current user question and the same or similar historical query request will be greatly increased or decreased. The system updates the feedback data corresponding to the current user question in the feedback confidence library. During the updating process, the system can combine and process the current query request and the same or similar historical query request. If there is no historical query request that is the same as or similar to the current query request, a new data is created in the feedback confidence library. The system updates and optimizes according to the information fed back by future query requests.

[0057] In one of the embodiments, the updating of the feedback confidence of each knowledge segment based on the feedback information comprises: increasing or decreasing the feedback confidence of the corresponding knowledge segment and the query request according to a third proportional coefficient based on the display feedback information of the knowledge segment retrieval result; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the query request according to a fourth proportional coefficient based on the implicit feedback information of the knowledge segment retrieval result; the third proportional coefficient is greater than the fourth proportional coefficient; that is, the weight of the display feedback information of the user to the knowledge segment retrieval result is greater than the weight of the implicit feedback information of the user to the knowledge segment retrieval result.

[0058] increasing or decreasing the feedback confidence of the corresponding knowledge segment and the query request according to a fifth proportional coefficient based on the display feedback information of the natural language answer; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the query request according to a sixth proportional coefficient based on the implicit feedback information of the natural language answer; the fifth proportional coefficient is greater than the sixth proportional coefficient. That is, the weight of the display feedback information of the user to the natural language answer is greater than the weight of the implicit feedback information of the user to the natural language answer.

[0059] In one of the embodiments, the user feedback module is further configured to: store a plurality of “query-knowledge segment” pairs based on the feedback information of the knowledge segment retrieval result and the generated natural language answer output by the user historical query request; and the knowledge retrieval module is specifically configured to: match the query request in each “query-knowledge segment” pair with the query request, if the matching is successful, directly output the corresponding knowledge segment retrieval result based on the matching result; if the matching is not successful, determine the text similarity, vector similarity and feedback confidence of each knowledge segment, and output the knowledge segment retrieval result based on the text similarity, vector similarity and feedback confidence of each knowledge segment.

[0060] In the embodiments of the present application, the user feedback module establishes a cache index for some queries and their corresponding high-confidence knowledge segments based on historical feedback records after they are verified multiple times. When encountering the same or highly similar queries subsequently, the system can directly return these knowledge segment results that have been tested by practice, greatly reducing repeated calculations and achieving significant improvement in retrieval speed, effectively solving the response delay bottleneck in enterprise-level applications. Under actual load, for common problems that hit the cache (about 40% of daily query volume), the average response time of the system is greatly reduced from 850 milliseconds to within 25 milliseconds. This not only greatly improves the user experience, but also effectively reduces the consumption of backend computing resources, enabling the system to more calmly cope with the challenges of enterprise-level massive knowledge bases and high-concurrency queries.

[0061] In one of the embodiments, the knowledge retrieval module is further configured to: weight and sum the text similarity, vector similarity and feedback confidence of each knowledge segment to obtain a total matching degree of each knowledge segment, and output a top-K matching knowledge segment retrieval result based on the total matching degree of each knowledge segment.

[0062] The embodiments of the present application also provide an enterprise knowledge Q&A method, which is applied to the system as described in any of the above embodiments, as shown in the figure, the method comprises the following steps: Figure 2

[0063] Step 201, based on the user's current query request, determine the text similarity, vector similarity and feedback confidence of the current query request and each knowledge segment, and output a knowledge segment retrieval result based on the text similarity, vector similarity and feedback confidence of each knowledge segment; wherein the feedback confidence is determined based on the feedback information of the user's output knowledge segment retrieval result and corresponding natural language answer for the previous query request;

[0064] Step 202, based on the current query request and the corresponding knowledge segment retrieval result, generate a natural language answer to the current query request using a large language model;

[0065] Step 203, collect the feedback information of the user's output knowledge segment retrieval result and natural language answer for the current query request, and update the feedback confidence of the corresponding knowledge segment and the current query request based on the feedback information.

[0066] Figure 3 is a flowchart of an enterprise knowledge Q&A method according to one of the embodiments, which comprises the following steps:

[0067] Step S301, split and label the enterprise original knowledge documents to obtain a plurality of knowledge sub-blocks (i.e. knowledge segments).

[0068] ​Step S302, using a pre-trained embedding model to convert the knowledge blocks into numerical vectors, and storing the embedded vectors and their corresponding original text fragments in a vector database.

[0069] Step S303, based on the vector database and the feedback confidence library, performing mixed retrieval on the user's question to obtain relevant knowledge fragments.

[0070] Step S304, based on the user's question and the relevant knowledge fragments, generating a natural language answer using a large language model.

[0071] Step S305, collecting feedback information of the user on the relevant knowledge fragments and the answer, and updating the feedback confidence of the relevant knowledge fragments according to the feedback information.

[0072] In an embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram thereof can be as shown in Figure 4 The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, mobile cellular network, NFC (Near Field Communication) or other technologies. The computer program is executed by the processor to implement an XXX method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad provided on the shell of the computer device. It can also be an external keyboard, touchpad or mouse, etc.

[0073] Those skilled in the art can understand that Figure 4 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0074] In an embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by the processor to implement the steps in any of the above enterprise knowledge question and answer method embodiments.

[0075] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0076] Any combination of the technical features of the above-mentioned embodiments can be combined. In order to make the description simple, all possible combinations of the technical features in the above-mentioned embodiments are not described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0077] The above-mentioned embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. An enterprise knowledge question answering system, characterized by, The system comprises: a knowledge base module configured to store a plurality of knowledge segments of an enterprise; a knowledge retrieval module configured to determine, based on a current query request of a user, a text similarity, a vector similarity and a feedback confidence of the current query request and each of the knowledge segments, and output a knowledge segment retrieval result based on the text similarity, the vector similarity and the feedback confidence of each of the knowledge segments; wherein the feedback confidence is determined based on feedback information of the user on a knowledge segment retrieval result and a corresponding natural language answer output for a previous query request; a response generation module configured to generate, based on the current query request and the corresponding knowledge segment retrieval result, a natural language answer to the current query request by using a large language model; a user feedback module configured to collect feedback information of the user on the knowledge segment retrieval result and the natural language answer output for the current query request, and update the feedback confidence of the corresponding knowledge segment and the current query request based on the feedback information.

2. The system of claim 1, wherein, The knowledge base module is specifically configured to: split each knowledge document of the enterprise to obtain a plurality of knowledge segments; add label information to each of the knowledge segments, convert each of the knowledge segments into a numerical vector, and store each of the knowledge segments and the corresponding numerical vector in a vector database.

3. The system of claim 1, wherein, The updating of the feedback confidence of the corresponding knowledge segment and the current query request based on the feedback information comprises: increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a first proportional coefficient based on the feedback information of the knowledge segment retrieval result; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a second proportional coefficient based on the feedback information of the natural language answer; the first proportional coefficient is greater than the second proportional coefficient.

4. The system of claim 1, wherein, The feedback information comprises implicit feedback information and explicit feedback information, the implicit feedback information comprises whether to click to browse or download a document containing the knowledge segment retrieval result and whether to click to copy the natural language answer, and the explicit feedback information comprises liking or disliking the knowledge segment retrieval result and the natural language answer.

5. The system of claim 4, wherein, The updating of the feedback confidence of each of the knowledge segments based on the feedback information comprises: increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a third proportional coefficient based on the explicit feedback information of the knowledge segment retrieval result; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a fourth proportional coefficient based on the implicit feedback information of the knowledge segment retrieval result; the third proportional coefficient is greater than the fourth proportional coefficient; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a fifth proportional coefficient based on the explicit feedback information of the natural language answer; increasing or decreasing the feedback confidence of the corresponding knowledge segment and the current query request by a sixth proportional coefficient based on the implicit feedback information of the natural language answer; the fifth proportional coefficient is greater than the sixth proportional coefficient.

6. The system of claim 1, wherein, The user feedback module is further configured to: retrieve results of knowledge segments based on the feedback information of the user on the previous query request and the generated natural language answer; The knowledge retrieval module is specifically configured to match the current query request with the query request in each of the "query-knowledge segment" pairs, and if the matching is successful, directly output the corresponding knowledge segment retrieval result based on the matching result; If the matching is not successful, determine the text similarity, vector similarity and feedback confidence of the current query request and each of the knowledge segments, and output the knowledge segment retrieval result based on the text similarity, vector similarity and feedback confidence of each of the knowledge segments.

7. The system of claim 1, wherein, The knowledge retrieval module is further configured to: weight and sum the text similarity, vector similarity and feedback confidence of each of the knowledge segments to obtain the total matching degree of each of the knowledge segments, and output the top K matching knowledge segment retrieval results based on the total matching degree.

8. An enterprise knowledge question answering method, characterized by, The method is applied to the system of any one of claims 1-7, and the method comprises: based on the current query request of the user, determine the text similarity, vector similarity and feedback confidence of the current query request and each of the knowledge segments, and output the knowledge segment retrieval result based on the text similarity, vector similarity and feedback confidence of each of the knowledge segments; wherein the feedback confidence is determined based on the feedback information of the user on the previous query request and the corresponding natural language answer; based on the current query request and the corresponding knowledge segment retrieval result, generate a natural language answer to the current query request using a large language model; collect the feedback information of the user on the knowledge segment retrieval result and the natural language answer of the current query request, and update the feedback confidence of the corresponding knowledge segment and the current query request based on the feedback information.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of claim 8. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-9. The processor executes the computer program to implement the method of claim 8.

Images