Intelligent knowledge recommendation method and device, readable storage medium and program product

Abstract

The invention discloses an intelligent knowledge recommendation method and device, a readable storage medium and a program product. The method comprises the steps of obtaining a target question of a target user; performing knowledge recommendation on the target question through a pre-trained natural language processing (NLP) model; if the NLP model does not recommend knowledge matched with the target question, performing retrieval enhancement generation on the target question through a pre-trained large language model (LLM); and based on a result generated by the retrieval enhancement, performing knowledge recommendation on the target question.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an intelligent knowledge recommendation method and apparatus, a readable storage medium, and a program product. Background Technology

[0002] In customer service scenarios across finance, telecommunications, and e-commerce sectors, both human and automated customer service representatives via online channels and hotlines need to provide timely and accurate knowledge recommendations based on user inquiries. Currently, intelligent knowledge recommendation methods include, for example, knowledge graph-based recommendations. This approach involves analyzing enterprise knowledge documents, extracting entities and relationships to construct a knowledge graph, and storing it in a graph database. When a user query is received, the system parses the real-time query and maps it to entities in the graph. Through deep graph traversal and relational reasoning, it discovers non-directly related knowledge paths, thereby generating recommendation results and pushing them to the corresponding user.

[0003] However, building intelligent knowledge recommendation systems based on knowledge graphs is extremely costly. Constructing and maintaining a high-quality, large-scale knowledge graph requires significant human resources (domain experts, knowledge engineers) and time for data annotation, model training, and manual review. Furthermore, it is highly domain-dependent; each vertical domain (such as finance, telecommunications, and hardware repair) requires a redesigned, unique knowledge graph, resulting in poor generality and high migration costs. In addition, knowledge graphs primarily store structured knowledge, while enterprise knowledge bases mostly contain unstructured content, potentially leading to insufficient knowledge coverage and reduced accuracy of recommendation results. Summary of the Invention

[0004] The purpose of this application is to provide an intelligent knowledge recommendation method and apparatus, and a readable storage medium, to solve the problems of high cost and low efficiency in existing knowledge graph-based knowledge recommendation.

[0005] To solve the above-mentioned technical problems, this specification is implemented as follows: Firstly, an intelligent knowledge recommendation method is provided, including: Obtain the target user's target question; Knowledge recommendation is performed on the target question using a pre-trained Natural Language Processing (NLP) model; If the NLP model does not recommend knowledge matching the target question, then the target question is retrieved and enhanced using a pre-trained Large Language Model (LLM). Based on the results generated by the enhanced retrieval, knowledge recommendations are made for the target question.

[0006] Optionally, the target question is enhanced and generated using a pre-trained Large Language Model (LLM), including: The target question is decomposed into multiple semantic units; The database queries the unstructured knowledge database based on the multiple semantic units. The unstructured knowledge database stores unstructured data including knowledge fragments with different semantics. The unstructured data is constructed based on data including unstructured fault reports, work order records, historical dialogue logs and long documents. The knowledge fragments matched by the multiple semantic units query are used as the context of the target question. The target question is then enhanced and generated using the LLM to obtain the result of the enhanced retrieval.

[0007] Optionally, knowledge recommendation is performed on the target question using a pre-trained Natural Language Processing (NLP) model, including: The NLP model is used to perform intent recognition on the target question to obtain the corresponding intent keywords. The target question includes the original question in the dialogue between the target user and the agent or a new question generated by rewriting the original question. The structured knowledge database is queried based on the intent keywords. The structured knowledge database stores structured data including different intent keywords and mappings. The structured data is constructed based on data including structured FAQ pairs, product manuals, and policy clauses. Based on the query results of the intent keywords, knowledge recommendations are made for the target question.

[0008] Optionally, based on the query results of the intent keywords, knowledge recommendation is performed on the target question, including: If the query result of the intent keyword is to find the target knowledge that matches the intent keyword and the corresponding matching confidence is higher than the preset confidence threshold, then the knowledge that matches the target question is recommended by the NLP model. If the query result for the intent keyword is that no target knowledge matching the intent keyword is found or the corresponding matching confidence is not higher than the preset confidence threshold, then the NLP model does not recommend knowledge matching the target question.

[0009] Optionally, the target user's target question is obtained, including: Obtain the original question from the dialogue between the target user and the agent as the target question; or Obtain the original questions asked during the dialogue between the target user and the agent; The NLP model is used to recommend knowledge from the original question. If the NLP model does not recommend knowledge matching the original question, the context of the original question is obtained through the LLM and the original question is rewritten in combination with the context to generate the target question.

[0010] Optionally, obtaining the context of the original question and rewriting the original question using the LLM in conjunction with the context to generate the target question includes: Obtain the most recent multi-turn dialogue as the context of the original question; Input the original question and the context into the LLM; By combining the LLM with the context, the original question is rewritten into one or more target questions with clear intent and standardized expression.

[0011] Optionally, before querying the unstructured knowledge database based on the plurality of semantic units, the method further includes: Collect data from target customer service scenarios, including unstructured fault reports, work order records, historical dialogue logs, and long documents. By parsing the collected data in different text formats, the text content is extracted; The text content is segmented into semantically complete knowledge fragments; The unstructured knowledge database is constructed based on the vectorized knowledge fragments.

[0012] In a second aspect, an intelligent knowledge recommendation device is provided, comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0013] Thirdly, a readable storage medium is provided that stores a program or instructions which, when executed by a processor, implement the steps of the method described in the first aspect.

[0014] Fourthly, a computer program product is provided, comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform the steps of the method described in the first aspect.

[0015] In this embodiment, the target user's question is obtained; a pre-trained Natural Language Processing (NLP) model is used to recommend knowledge from the target question; if the NLP model does not recommend knowledge matching the target question, a pre-trained Large Language Model (LLM) is used to perform retrieval enhancement generation on the target question; based on the results of the retrieval enhancement generation, knowledge is recommended from the target question, thus adopting a staged architecture where small and large models work together. The small model (NLP model) is responsible for routine intent recognition and preliminary knowledge recommendation, while the large model (LLM model) intervenes when the small model cannot handle the task effectively, performing deep knowledge retrieval and generation. The low-cost, fast-response small model handles most routine and explicit user questions, while the powerful but costly large model is called only as needed for complex, ambiguous, or rare questions to perform RAG generation. This ensures a high recommendation success rate and high answer quality while significantly reducing the system's average response latency and computational resource consumption, achieving a balance between effectiveness, cost, and real-time performance. Compared to traditional knowledge recommendation based on knowledge graphs, this application avoids the problems of extremely high construction costs, strong domain dependence, and inability to handle unstructured knowledge. It has the advantages of lower construction costs, shorter cycle, and stronger versatility, and can quickly adapt to new businesses and new fields. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating the intelligent knowledge recommendation method according to an embodiment of this application.

[0017] Figure 2 This is a schematic diagram of the hybrid knowledge database construction process according to an embodiment of this application.

[0018] Figure 3 This is a schematic diagram of the overall process of an intelligent knowledge recommendation method according to a specific embodiment of this application.

[0019] Figure 4 This is a schematic diagram of the overall process of an intelligent knowledge recommendation method according to another specific embodiment of this application.

[0020] Figure 5 This is a structural block diagram of the intelligent knowledge recommendation device according to an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. The drawing numbers in this application are only used to distinguish the various steps in the solution and are not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0022] To address the problems existing in the prior art, embodiments of this application provide an intelligent knowledge recommendation method, such as... Figure 1 As shown, the process includes steps 102 to 108.

[0023] Step 102: Obtain the target user's target question.

[0024] The target users are customers in customer service scenarios in fields such as finance, telecommunications, or e-commerce who conduct real-time inquiry through online or hotline channels. The intelligent knowledge recommendation method in this application is applied to the corresponding customer service system, providing intelligent knowledge recommendations based on each user's real-time inquiry.

[0025] The target question is a question related to the real-time query of the target user. The target user can form the corresponding real-time query question through online text input or hotline voice input.

[0026] Based on the solution provided in the above embodiments, optionally, in step 102 above, obtaining the target question of the target user includes: obtaining the original question in the dialogue between the target user and the agent as the target question.

[0027] During the conversation between the customer service agent and the target user, a dialogue stream is generated. The target user's current question, i.e. the original question, is obtained in real time from the dialogue stream.

[0028] Step 104: Use a pre-trained Natural Language Processing (NLP) model to recommend knowledge about the target question.

[0029] Natural Language Processing (NLP) models are lightweight artificial intelligence technologies capable of understanding, analyzing, processing, and generating human language. NLP models use structured knowledge, such as FAQs compiled from enterprise knowledge documents, as samples. For each sample's knowledge content, a large number of rules, including keywords, regular expressions, and similarity questions, are configured. Each sample's corresponding knowledge is then labeled to train the NLP model.

[0030] Based on the solution provided in the above embodiments, optionally, in step 104 above, knowledge recommendation for the target question using a pre-trained Natural Language Processing (NLP) model includes: using the NLP model to perform intent recognition on the target question to obtain corresponding intent keywords, wherein the target question includes the original question in the dialogue between the target user and the agent or a new question generated by rewriting the original question; querying a structured knowledge database based on the intent keywords, wherein the structured knowledge database stores structured data including different intent keywords and mappings, wherein the structured data is constructed based on data including structured FAQ pairs, product manuals, and policy clauses; and recommending knowledge for the target question based on the query results of the intent keywords.

[0031] In this embodiment, a lightweight NLP model is used to identify the target user's question intent in real time and obtain the corresponding intent keywords. Specifically, the NLP model uses NLP techniques such as lexical analysis, syntactic analysis, and semantic understanding to extract keywords or key information from the target question for intent identification and obtain the corresponding intent keywords.

[0032] The target question includes the original question during the dialogue between the target user and the agent, or a new question generated by rewriting the original question (the new question will be described below). Intent keywords are used to trigger pre-defined keyword or similar question rules in the NLP model. Each rule is pre-configured with corresponding knowledge content. That is, through the rules associated with the intent keywords and the knowledge associated with those rules, a structured knowledge database is queried to match relevant knowledge points in the structured knowledge database, generating recommended knowledge and returning it to the target user.

[0033] The structured knowledge database stores structured data containing knowledge with different intent keywords and mappings. This structured data is built upon data including structured FAQ pairs, product manuals, and policy clauses. Therefore, knowledge recommendations can be made for target questions based on the query results of intent keywords.

[0034] Specifically, the construction process of a structured knowledge database is as follows: Figure 2 As shown, Figure 2 This is a schematic diagram of the hybrid knowledge database construction process according to an embodiment of this application. Here, the hybrid knowledge database refers to a structured knowledge database and an unstructured knowledge database (the unstructured knowledge database will be described below).

[0035] like Figure 2As shown, the construction process of the structured knowledge database is as follows: Multi-source knowledge collection and classification (step 202): Collect and organize various types of knowledge data required in enterprise customer service scenarios, including structured data, such as frequently asked questions (FAQ) pairs, product manuals, and policy clauses; Preprocess the structured data such as FAQs (step 204), such as cleaning, labeling, and classification, to clarify the intent classification and keywords corresponding to each piece of knowledge, obtain structured knowledge, and store it in a relational database such as MySQL, and establish an efficient index to complete the construction of the structured knowledge database (step 214).

[0036] Specifically, based on the query results of the intent keywords, knowledge recommendation is performed on the target question, including: if the query result of the intent keywords is that target knowledge matching the intent keywords is found and the corresponding matching confidence is higher than a preset confidence threshold, then the knowledge matching the target question is recommended through the NLP model; if the query result of the intent keywords is that target knowledge matching the intent keywords is not found or the corresponding matching confidence is not higher than the preset confidence threshold, then the knowledge matching the target question is not recommended through the NLP model.

[0037] When performing structured knowledge database queries based on intent keywords obtained from intent recognition of target questions using NLP models, if the intent keywords trigger preset keyword or similar question rules in the NLP model, knowledge recommendations are made according to the knowledge content configured for the matching rule and the corresponding matching confidence score. If the matching confidence score corresponding to the target rule is higher than the preset confidence threshold, the knowledge content corresponding to the target rule is used as the query result, knowledge recommendations are made for the target question, and pushed to the target user.

[0038] If the intent keyword does not trigger any of the preset keyword or similar question rules in the NLP model, that is, the query result of the intent keyword is that no matching target knowledge is found, or if the matching confidence of each rule triggered by the intent keyword is not higher than the preset confidence threshold, it means that the NLP model cannot recommend knowledge matching the target question, and the recommendation fails.

[0039] During NLP model training, business-accumulated dialogue data is used for intent recognition, and the model output is a preset intent. An efficient, precise matching module is built based on similar question corpora and regular rules to supplement the model's retrieval. The training goal is to enable the NLP model to quickly and accurately identify preset common intents from user questions.

[0040] Step 106: If the NLP model does not recommend knowledge matching the target question, then the target question is retrieved and enhanced using a pre-trained Large Language Model (LLM).

[0041] When the NLP model fails to provide knowledge matching the target question, a pre-trained Large Language Model (LLM) is used to enhance the generation of the target question. LLM refers to a deep learning model trained on large amounts of text data, enabling it to generate natural language text or understand the meaning of language text. LLM models can provide in-depth knowledge and language production on a wide range of topics by training on massive datasets.

[0042] Retrieval-augmented generation (RAG) is a cutting-edge technique in large-scale modeling, combining language modeling and information retrieval techniques. Specifically, when an LLM model needs to answer a target question, it first retrieves relevant information from a large collection of documents, and then uses this retrieved information to guide the generation of an answer to the question, thereby improving the quality and accuracy of the answer.

[0043] Based on the solution provided in the above embodiments, optionally, in step 106 above, the retrieval enhancement generation of the target question using a pre-trained Large Language Model (LLM) includes: decomposing the target question into multiple semantic units; querying an unstructured knowledge database based on the multiple semantic units, wherein the unstructured knowledge database stores unstructured data including knowledge fragments with different semantics, and the unstructured data is constructed based on data including unstructured fault reports, work order records, historical dialogue logs, and long documents; using the knowledge fragments matched by the query of the multiple semantic units as the context of the target question, and performing retrieval enhancement generation of the target question using the LLM to obtain the result of the retrieval enhancement generation.

[0044] In this application, during hotline or online service, the NLP model (i.e., the small model) is first triggered to perform initial processing on the questions raised by the target user. The LLM model (i.e., the large model) then intervenes when the small model cannot effectively handle the questions, and performs deep knowledge retrieval and enhancement generation based on the target questions.

[0045] Specifically, during real-time dialogue, the target user's question is parsed and segmented into multiple semantic units, which are then vectorized. Based on the vector values ​​corresponding to each semantic unit, similarity retrieval is performed in an unstructured knowledge database (or vector database) containing unstructured data vector values ​​of knowledge fragments with different semantic meanings, thereby obtaining knowledge fragments related to these semantic units.

[0046] By calculating the semantic similarity represented by corresponding vector values, the top-K knowledge fragments most similar to the target question in the unstructured knowledge database are selected. The LLM model uses the retrieved most similar knowledge fragments as the context of the target question, forming a prompt together with the target question. Combining its own language generation capabilities, the model integrates these retrieved fragmented knowledge fragments to generate a coherent, accurate, and complete knowledge answer to the question corresponding to the target question, resulting in an enhanced retrieval result that is returned to the target user.

[0047] By designing high-quality Prompt projects, LLM models can be trained to utilize retrieved document fragments as context to generate concise, accurate, and professional answers, while strictly avoiding illusions. The Prompt should explicitly restrict the model to answering solely based on retrieved knowledge, thereby optimizing RAG generation.

[0048] The unstructured data is constructed based on unstructured fault reports, work order records, historical dialogue logs, and long documents. In one specific embodiment, before querying the unstructured knowledge database based on the multiple semantic units, the process further includes: collecting data from the target customer service scenario, including unstructured fault reports, work order records, historical dialogue logs, and long documents; parsing the collected data in different text formats to extract text content; segmenting the text content into semantically complete knowledge fragments; and constructing the unstructured knowledge database based on the vectorized knowledge fragments.

[0049] Specifically, the construction process of a structured knowledge database is as follows: Figure 2 As shown, Figure 2 This is a schematic diagram of the hybrid knowledge database construction process according to an embodiment of this application. Here, the hybrid knowledge database refers to a structured knowledge database and an unstructured knowledge database (the unstructured knowledge database will be described below).

[0050] like Figure 2As shown, the construction process of the unstructured knowledge database is as follows: Multi-source knowledge collection and classification (step 202): Collect and organize various types of knowledge data required in enterprise customer service scenarios, including unstructured data such as fault reports, work order records, historical dialogue logs, long documents, etc.; Preprocess the unstructured data to obtain unstructured documents (step 204); Parse unstructured documents in Word, PDF, TXT and other formats (step 206) to extract plain text content; Use a text segmentation algorithm to divide the extracted long text into multiple semantically complete knowledge fragments (step 208); Use an embedding algorithm to vectorize the knowledge fragments (step 210); Store the vectorized knowledge fragments and their metadata in a vector database (step 212), such as the unstructured knowledge database of FAISS, to complete the knowledge reserve. The unstructured knowledge database is used to store and manage a large amount of unstructured document data and supports RAG retrieval of the LLM model.

[0051] Step 108: Based on the results generated by the retrieval enhancement, knowledge recommendation is performed on the target question.

[0052] As described above, in one embodiment, the target question includes the original question during the dialogue between the target user and the agent. In this embodiment, it is combined with... Figure 3 It can be seen that the specific process of intelligent knowledge recommendation includes the following steps: Step 302: During hotline or online service, obtain the conversation flow between the user and the agent; Step 304: In response to the original question posed by the target user, the first stage is triggered: small model, i.e., intent recognition of the NLP model, to obtain the corresponding intent keywords; Step 306: Compare the matching confidence of the query results obtained from the structured knowledge database of intent keyword query with the preset matching confidence to determine whether there are query results and whether the query results correspond to high matching confidence results. If yes, proceed to step 308; otherwise, proceed to step 318. Step 308: Recommend knowledge matching the original question. Recommendation successful. The intelligent knowledge recommendation process for the original question ends. Step 318: In response to the original question raised by the target user, the second stage is triggered: large model, namely the RGA retrieval generation of the LLM model; Step 320: Generate the final answer to the original question based on the results of the retrieval enhancement generation, and recommend it to the original question; Step 322: The intelligent knowledge recommendation process for the original question ends.

[0053] In other words, for the original question posed by the target user, the NLP model is first triggered to recommend knowledge. If the NLP model does not recommend knowledge that matches the original question, the LLM model intervenes, performs retrieval enhancement generation on the original question, and recommends the corresponding knowledge to the target user's original question based on the retrieval enhancement generation results.

[0054] In the embodiments described above, a lightweight NLP model is first invoked for intent recognition. This small model performs a fast matching retrieval in a structured knowledge base (e.g., a MySQL database). If knowledge with a confidence level higher than a threshold (e.g., 90%) is matched, it is immediately recommended to the target user, and the process ends. This stage typically takes milliseconds, ensuring real-time performance. If the NLP model fails to match or has low confidence, an LLM model is invoked for retrieval enhancement and generation, ensuring a high recommendation success rate and high answer quality.

[0055] In another embodiment, the target question includes a new question generated by rewriting the original question. Correspondingly, obtaining the target user's target question includes: obtaining the original question during the dialogue between the target user and the agent; performing knowledge recommendation on the original question using the NLP model; if the NLP model does not recommend knowledge matching the original question, then obtaining the context of the original question using the LLM and rewriting the original question in combination with the context to generate the target question.

[0056] In this embodiment, Figure 3 After step 306, which uses the NLP model to determine that there is no corresponding high-confidence matching result for the query (i.e., the recommendation is unsuccessful), and before step 318, which uses the LLM model to generate the retrieval, the original question is rewritten using the LLM model to generate a new question, and the NLP model is triggered again to recommend knowledge based on the new question.

[0057] Specifically, obtaining the context of the original question and rewriting the original question using the LLM in conjunction with the context to generate the target question includes: obtaining the most recent multi-turn dialogue of the original question as the context of the original question; inputting the original question and the context into the LLM; and performing referential resolution, entity completion, and / or semantic generalization on the original question using the LLM in conjunction with the context to rewrite the original question into one or more target questions with clear intent and standardized expression.

[0058] If the NLP model does not match the query result or the confidence of the query result is low, the original question of the target user and its most recent dialogue (e.g., the first three rounds of dialogue) are sent to the LLM model as the context of the original question.

[0059] LLM models, leveraging powerful language understanding and generation capabilities, analyze and rewrite original questions in conjunction with context. By performing referential resolution, entity completion, and / or semantic generalization on the original questions, they generate clearer and more accurate new questions, outputting one or more rewritten, clearly defined "standard questions." For example, if the target user's original question is vague, "How do I use your product?", the LLM model can rewrite it as, "How do I use your company's product model * with ** as its main feature?"

[0060] By using paired data containing ambiguous questions and their corresponding clear questions, the LLM model is supervised fine-tuned (SFT) to enable it to learn to resolve referential differences, complete entities, and reconstruct questions based on the dialogue context. For example, "How do I activate that service?" can be rewritten in context as "How do I activate the 5G Enjoy Service Package?", thus optimizing the LLM model's capabilities.

[0061] Regarding dialogue context management, a dynamic dialogue state table can be maintained throughout the multi-turn dialogue process, recording the current dialogue topic, mentioned entities (such as product names and order numbers), and recommendation history. This state is used to assist the LLM model in rewriting context, ensuring the accuracy of rewriting new questions.

[0062] The LLM model feeds the rewritten new question back into the NLP model for intent recognition and retrieval, greatly improving the success rate of the subsequent NLP model in recognizing the user's true intent. This activates the reuse value of the existing structured knowledge base and serves as a key bridge connecting the lightweight processing of the NLP model with the heavyweight processing of the LLM model.

[0063] In this embodiment, combined with Figure 4 It can be seen that the specific process of intelligent knowledge recommendation includes the following steps: Step 402: During hotline or online service, obtain the conversation flow between the user and the agent; Step 404: In response to the original question posed by the target user, the first stage is triggered: small model, i.e., intent recognition of the NLP model, to obtain the corresponding intent keywords; Step 406: Compare the matching confidence of the query results obtained from the structured knowledge database of intent keyword query with the preset matching confidence to determine whether there are query results and whether the query results correspond to high matching confidence results. If yes, proceed to step 408; otherwise, proceed to step 410. Step 408: Recommend knowledge matching the original question. Recommendation successful. The intelligent knowledge recommendation process for the original question ends. Step 410: In response to the original question raised by the target user, the second stage is triggered: the large model, i.e., the LLM model rewrites the original question to obtain a new question; Step 412: For the new question rewritten by the LLM model, the small model, i.e. the intent recognition of the NLP model, is triggered again to obtain the corresponding intent keywords. Step 414: Compare the matching confidence of the query results obtained from the structured knowledge database by querying the intent keyword with the preset matching confidence to determine whether the query results correspond to high matching confidence results. If yes, proceed to step 416; otherwise, proceed to step 418. Step 416: Recommend knowledge matching the new question. Recommendation successful. The intelligent knowledge recommendation process for the original question ends. Step 418: For the new question, the third stage is triggered: large model, i.e., RGA retrieval generation of the LLM model; Step 420: Generate the final answer to the new question based on the results of the retrieval enhancement generation, and recommend it to the original question; Step 422: The intelligent knowledge recommendation process for the original question ends.

[0064] In other words, for the original question posed by the target user, the NLP model is first triggered to recommend knowledge. If the NLP model does not recommend knowledge matching the original question, the LLM model intervenes to rewrite the original question. Then, the NLP model is triggered again to recommend knowledge. If the NLP model does not recommend knowledge matching the new question, the LLM model intervenes again to perform retrieval enhancement generation on the new question, and based on the retrieval enhancement generation results of the new question, recommend the corresponding knowledge to the target user's original question.

[0065] This embodiment proposes a phased model-coordinated intelligent recommendation mechanism: a progressive triggering and decision-making process consisting of "small model priority processing → large model context rewriting → small model retrying → large model RAG generation". Its core lies in using low-cost, fast-response small models to handle the vast majority of regular and explicit user questions, only calling powerful but costly large models for in-depth processing (semantic rewriting or RAG generation) when dealing with complex, ambiguous, or rare questions. This mechanism significantly reduces the system's average response latency and computational resource consumption while ensuring high recommendation success rate and high answer quality, achieving a balance between effectiveness, cost, and real-time performance.

[0066] Furthermore, for the structured and unstructured knowledge databases in this application embodiment, a dynamic knowledge update mechanism can be used for updating. Specifically, (1) Structured knowledge update: Administrators can add, modify, or delete FAQ entries through the backend interface, and the update will take effect immediately in the NLP model's retrieval database. (2) Unstructured document update: When a new policy document or fault report is generated, it is automatically parsed, segmented, and vectorized, and incrementally updated to the vector database without retraining the model, thus realizing the dynamic and low-cost expansion of the knowledge of the LLM model.

[0067] This application's embodiments construct and collaboratively utilize structured knowledge bases (such as MySQL) and unstructured vector document libraries (such as those based on FAISS). The structured knowledge base supports NLP models for efficient and accurate intent matching; the unstructured vector library supports LLM models for extensive semantic retrieval. Through the aforementioned phased process, the two are organically integrated to form a comprehensive knowledge foundation (from precise answers to fragmented knowledge) that supports low-cost dynamic updates (especially for unstructured knowledge), fundamentally overcoming the limitations of a single type of knowledge base.

[0068] In this embodiment, the target user's question is obtained; a pre-trained Natural Language Processing (NLP) model is used to recommend knowledge from the target question; if the NLP model does not recommend knowledge matching the target question, a pre-trained Large Language Model (LLM) is used to perform retrieval enhancement generation on the target question; based on the results of the retrieval enhancement generation, knowledge is recommended from the target question, thus adopting a staged architecture where small and large models work together. The small model (NLP model) is responsible for routine intent recognition and preliminary knowledge recommendation, while the large model (LLM model) intervenes when the small model cannot handle the task effectively, performing deep knowledge retrieval and generation. The low-cost, fast-response small model handles most routine and explicit user questions, while the powerful but costly large model is called only as needed for complex, ambiguous, or rare questions to perform RAG generation. This ensures a high recommendation success rate and high answer quality while significantly reducing the system's average response latency and computational resource consumption, achieving a balance between effectiveness, cost, and real-time performance. Compared to traditional knowledge recommendation based on knowledge graphs, this application avoids the problems of extremely high construction costs, strong domain dependence, and inability to handle unstructured knowledge. It has the advantages of lower construction costs, shorter cycle, and stronger versatility, and can quickly adapt to new businesses and new fields.

[0069] In summary, the embodiments of this application, based on a phased model collaboration-based intelligent knowledge recommendation, achieve an efficient, accurate, reliable, universal, and low-cost knowledge recommendation service.

[0070] The applicable business scenarios for this application's embodiments are as follows: In customer service scenarios in fields such as finance, telecommunications, and e-commerce, during customer service through hotline channels and online channels, the system automatically provides intelligent knowledge recommendation services, pushing accurate knowledge cards to the customer service personnel's chat console interface. Customer service personnel do not need to manually search the knowledge base; they only need to click "one-click send" or directly read the recommended content aloud, helping new agents quickly get up to speed, efficiently and accurately answer customer questions, and reduce call duration. Alternatively, accurate knowledge cards can be pushed to a chatbot for answering customer questions.

[0071] The summarized business value includes: 1. Enhanced customer experience: Rapid and accurate response to needs, reduced waiting time, and increased customer satisfaction and loyalty, especially effective in service-sensitive sectors such as finance and electricity; 2. Reduced enterprise costs: Reduced labor training costs and the number of agents, and increased efficiency per agent; 3. Increased sales conversion rate: Precise product recommendations and sales script guidance can increase the conversion rate by more than 20% in sales scenarios such as wealth management and insurance; 4. Expanded application scenarios: Applicable to multiple industries requiring a large number of agents, such as finance, telecommunications, and e-commerce, with broad market application prospects. Enterprises can enhance their competitiveness and gain an advantage in the market by introducing this technology.

[0072] Optionally, such as Figure 5 As shown in the figure, this application embodiment also provides an intelligent knowledge recommendation device 2000, including a processor 2400 and a memory 2200. The memory 2200 stores a program or instructions that can run on the processor 2400. When the program or instructions are executed by the processor 2400, they implement the various steps of the above-described intelligent knowledge recommendation method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0073] This application also provides a readable storage medium storing a program or instructions. When executed by a processor, the program or instructions implement the various processes of any of the above-described intelligent knowledge recommendation method embodiments and achieve the same technical effect. To avoid repetition, further details are omitted here. The readable storage medium includes computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. The computer program is operable to enable a computer to execute various processes of any of the above-described intelligent knowledge recommendation method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0075] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0076] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0077] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An intelligent knowledge recommendation method, characterized in that, include: Obtain the target user's target question; Knowledge recommendation is performed on the target question using a pre-trained Natural Language Processing (NLP) model; If the NLP model does not recommend knowledge matching the target question, then the target question is retrieved and enhanced using a pre-trained Large Language Model (LLM). Based on the results generated by the enhanced retrieval, knowledge recommendations are made for the target question.

2. The method according to claim 1, characterized in that, The target question is enhanced and generated using a pre-trained Large Language Model (LLM), including: The target question is decomposed into multiple semantic units; The database queries the unstructured knowledge database based on the multiple semantic units. The unstructured knowledge database stores unstructured data including knowledge fragments with different semantics. The unstructured data is constructed based on data including unstructured fault reports, work order records, historical dialogue logs and long documents. The knowledge fragments matched by the multiple semantic units query are used as the context of the target question. The target question is then enhanced and generated using the LLM to obtain the result of the enhanced retrieval.

3. The method according to claim 1 or 2, characterized in that, Knowledge recommendation is performed on the target question using a pre-trained Natural Language Processing (NLP) model, including: The NLP model is used to perform intent recognition on the target question to obtain the corresponding intent keywords. The target question includes the original question in the dialogue between the target user and the agent or a new question generated by rewriting the original question. The structured knowledge database is queried based on the intent keywords. The structured knowledge database stores structured data including different intent keywords and mappings. The structured data is constructed based on data including structured FAQ pairs, product manuals, and policy clauses. Based on the query results of the intent keywords, knowledge recommendations are made for the target question.

4. The method according to claim 3, characterized in that, Based on the query results of the intent keywords, knowledge recommendation is performed on the target question, including: If the query result of the intent keyword is to find the target knowledge that matches the intent keyword and the corresponding matching confidence is higher than the preset confidence threshold, then the knowledge that matches the target question is recommended by the NLP model. If the query result for the intent keyword is that no target knowledge matching the intent keyword is found or the corresponding matching confidence is not higher than the preset confidence threshold, then the NLP model does not recommend knowledge matching the target question.

5. The method according to claim 3, characterized in that, Obtain the target user's target questions, including: Obtain the original question from the dialogue between the target user and the agent as the target question; or Obtain the original questions asked during the dialogue between the target user and the agent; The NLP model is used to recommend knowledge from the original question. If the NLP model does not recommend knowledge matching the original question, the context of the original question is obtained through the LLM and the original question is rewritten in combination with the context to generate the target question.

6. The method according to claim 5, characterized in that, Obtaining the context of the original question and rewriting it using the LLM in conjunction with the context to generate the target question includes: Obtain the most recent multi-turn dialogue as the context of the original question; Input the original question and the context into the LLM; By combining the LLM with the context, the original question is rewritten into one or more target questions with clear intent and standardized expression.

7. The method according to claim 2, characterized in that, Before querying the unstructured knowledge database based on the aforementioned multiple semantic units, the process also includes: Collect data from target customer service scenarios, including unstructured fault reports, work order records, historical dialogue logs, and long documents. By parsing the collected data in different text formats, the text content is extracted; The text content is segmented into semantically complete knowledge fragments; The unstructured knowledge database is constructed based on the vectorized knowledge fragments.

8. An intelligent knowledge recommendation device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the method as described in any one of claims 1-7.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform the steps of the method as described in any one of claims 1-7.

Images