Intelligent question-answering multi-knowledge-base classification optimization method and system for power industry

Abstract

The invention discloses a power industry intelligent question-answering multi-knowledge-base classification optimization method and system, and relates to the field of power industry knowledge base optimization. In order to solve the defects that an existing knowledge base is not suitable for a power industry knowledge base, the classification accuracy is poor, and retrieval results and answer quality are poor, the power knowledge base is classified according to division of actual business of a power enterprise to form a plurality of knowledge bases; outputting an initial result through a large language model; collecting feedback of the user and carrying out error tracing positioning on negative feedback; constructing a case library and performing classified management on cases; according to a multi-source classification verification result, storing a query fed back by a user into a corresponding case library; the weights of the case query are calculated and sorted; and designing and generating a classification prompt word structure, and guiding the large language model to perform classification optimization on query of the user. The method is mainly used for optimizing intelligent question and answer multi-knowledge-base classification in the power industry.

Description

Technical Field

[0001] The present invention relates to the field of knowledge base optimization in the electric power industry, and in particular to a classification optimization method and system for intelligent question-answering multi-knowledge bases in the electric power industry based on user feedback. Background Art

[0002] In recent years, intelligent question-answering systems that combine large language models (LLM) with retrieval-augmented generation (RAG) technology have been widely used in various industries. However, for some special industries, such as the power industry, due to the complex and huge knowledge system of the power industry knowledge base, a single database is difficult to effectively cover the retrieval needs and knowledge storage of all fields. Therefore, building multiple knowledge bases by professional division has become a more efficient way of knowledge management.

[0003] In the multi-knowledge base intelligent question answering scenario, the user's query needs to be classified according to its semantic content first to determine which knowledge base to retrieve content from to support the answer to the question. Therefore, the classification accuracy of the user's question directly determines the system's retrieval effect and answer quality.

[0004] Although the existing large language models have powerful natural language understanding capabilities, query classification can be achieved through prompt word design or model fine-tuning. However, the diversity of user questions, the flexibility of language expression, and the complexity of business knowledge will affect the accuracy of question classification. And there are still many problems when fine-tuning the model. For example, a large number of labeled training samples are required, hardware resource consumption is high, and the fine-tuning model is difficult to adapt to the dynamic changes in the distribution of user questions in a timely manner. The above problems limit the large-scale application of fine-tuning methods in multi-knowledge base intelligent question-answering systems in the power industry.

[0005] Therefore, there is a need for an intelligent question-answering multi-knowledge base classification optimization method that can be applied to the knowledge base of the power industry, with high classification accuracy, precise retrieval results, high answer quality and good flexibility. Summary of the invention

[0006] In order to solve the defects that the existing knowledge base cannot be applied to the knowledge base of the electric power industry, has poor classification accuracy, and poor retrieval results and answer quality, the present invention provides an intelligent question and answer multi-knowledge base classification optimization method that can be applied to the knowledge base of the electric power industry, has high classification accuracy, precise retrieval results, high answer quality, and good flexibility.

[0007] The present invention provides a method for optimizing classification of multiple knowledge bases for intelligent question answering in the electric power industry, comprising the following steps: S1. Classify the power knowledge base according to the actual business division of the power enterprise to form multiple knowledge bases; S2, output the initial result through the large language model; S3, collect user feedback and trace the source of errors for negative feedback; S4. Build a case library and classify and manage cases; According to the multi-source classification verification results, the queries with user feedback are stored in the corresponding case library; S5, calculating and ranking the weights of case queries; S6. Design and generate classification prompt word structures to guide the large language model to classify and optimize user queries.

[0008] Further: in S2, the initial result includes an initial classification result and an initial answer; the specific steps of outputting the initial result through the large language model are as follows: S21. Query reception and classification: Receive the query input by the user and submit it to the large language model for classification; S22. Classification confidence distribution calculation: The large language model generates the confidence level of the query in each knowledge base. distributed ,in The query belongs to the knowledge base Confidence level; is the number of knowledge bases; S23, classification results are determined: Determine the initial classification result according to the classification confidence , , that is, select the knowledge base with the highest confidence ; S24, initial answer generation: From the initial classification results Corresponding knowledge base Search Related content snippets , generating an initial answer through the large language model and save it.

[0009] Further: In S3, the specific steps of collecting user feedback and tracing the source of errors for negative feedback are as follows: S31. Obtain user feedback: Get the user's initial answer to the Feedback, and divide all feedback into positive feedback and negative feedback; S32. Multi-source classification verification mechanism: For the query corresponding to the negative feedback and the initial answer , trace the root cause of the negative feedback, that is, whether the negative feedback is caused by errors in the problem classification link.

[0010] Further: in S32, tracing the root cause of the negative feedback specifically includes the following steps: S321. Multiple knowledge base search answers: Send the query to all knowledge bases separately , retrieve content fragments, and generate a set of candidate answers ; S322, Answer Evaluation: For the candidate answer set Perform quality assessment and collect query and all knowledge bases Retrieve relevant content snippets from the search results and select the best answer , and its corresponding knowledge base is ; S323. Verification of classification results: If classification , then the classification is judged to be wrong, if , the classification is correct.

[0011] Further: in S5, the specific steps of calculating and ranking the weights of the case queries are as follows: S51. Case weight calculation: Calculate weights for case queries , the weight The calculation formula is: (1) in, is the time factor, is the classification confidence factor, is the knowledge base coverage factor, is the information entropy, is the total weight; Refers to the time factor coefficient, Refers to the classification confidence factor coefficient, refers to the knowledge base coverage factor coefficient, refers to information entropy, and ; S52, Redundancy Detection and Optimization: Calculate the similarity between the case and other cases. If the similarity is greater than the preset threshold, delete the cases whose weight is lower than the preset threshold. S53. Case weight ranking: After removing redundancy, the positive and negative case sets are sorted according to their weights.

[0012] Further: in S6, the specific steps of designing and generating the classification prompt word structure are as follows: S61, preset classification guidelines; S62, construct prompt words; Design the structure of the classification prompt words according to the task description, positive and negative case presentations, and classification guidelines; S63. Prompt word generation and deployment.

[0013] Further: in S61, the preset classification guideline specifically includes the following steps: S611, Domain Attribution Analysis: Divide the responsibilities of each business domain, clarify the core focus of each domain, map the core focus to the knowledge base category, and form the "core business domain distinction" rule; S612, feature word extraction: by analyzing historical classification data, extracting keywords from different knowledge bases according to the degree of discrimination, and organizing them into a category keyword list to form a "keyword feature recognition" rule; S613, scenario divergence resolution: for scenarios that are confusing in the classification history, summarize the reasons for the fuzzy classification boundaries, and formulate a judgment strategy based on the priority of focus points in combination with business logic and actual needs to form a "principle for handling easily confusing scenarios"; S614, semantic context modeling: Utilize the context and query background, combined with a complete semantic understanding model or rules, to clarify the real business needs of classification, ensure the accuracy of classification decisions, and form a "context comprehensive judgment" rule.

[0014] Further: in S63, the prompt word generation and deployment specifically includes the following steps: S631, positive and negative case insertion rules: insert cases alternately according to the preset positive and negative case ratio to ensure that the classification rules are fully covered; S632, prompt word verification: Use the test set to verify the classification accuracy. If the classification accuracy meets the standard, the online test is passed; S633, prompt words online: deploy the generated prompt words to the classification module and enable real-time classification service.

[0015] The classification optimization system of the intelligent question-answering multi-knowledge base classification optimization method for the power industry described in the present invention includes a knowledge base classification module, a large language model, a feedback processing module, a classification management module, a weight calculation module and a classification prompt word construction module; The knowledge base classification module is used to classify the power knowledge base according to the actual business division of the power enterprise to form multiple knowledge bases; The large language model is used to answer the query and output an initial result; The feedback processing module is used to collect user feedback and perform error tracing and location for negative feedback; The classification management module is used to build a case library and classify and manage cases; The weight calculation module is used to calculate and sort the weights of case queries; The classification prompt word construction module is used to design and generate the classification prompt word structure.

[0016] The beneficial effects of the present invention are: The present invention discloses a method and system for optimizing classification of multi-knowledge bases for intelligent question and answer in the electric power industry based on user feedback, wherein prompt word engineering is a method that does not require model parameter updates, and guides the large language model to generate higher quality outputs by designing reasonable input content, such as typical cases, reasoning steps, etc. It has the advantages of low cost, no need for a large amount of labeled data, and flexibility, thereby improving the classification performance of large models. After the intelligent question and answer system is launched, the user's real feedback (such as likes, dislikes, etc.) can provide valuable data support for the classification optimization based on prompt words. By collecting and utilizing user feedback, a positive and negative case library can be constructed, and the content of the classification prompt words can be dynamically adjusted and optimized based on these cases and the reasoning basis of the cases, thereby stimulating the large language model to complete the classification task more accurately, which helps to improve the overall performance of the intelligent question and answer system.

[0017] The intelligent question-answering multi-knowledge base classification optimization method based on user feedback described in the present invention makes full use of user feedback information to improve the model classification prompt words, and proposes strategies such as multi-dimensional case weight calculation, multi-source classification verification (MSCV, Multi-Source Classification Validation), and construction of positive and negative case libraries with reasoning basis to improve the actual application effect of the present invention in many aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart for constructing classification prompt words; Figure 2 It is a knowledge base classification and optimization flow chart. DETAILED DESCRIPTION

[0019] The following are only preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a technician familiar with the technical field within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. The embodiments described below are only used to explain the present invention and cannot be interpreted as limitations on the present invention. The protection scope of the present invention should be based on the protection scope of the claims. The embodiments of the present invention are described in detail below. In order to facilitate the description of the present invention and simplify the description, the technical terms used in the specification of the present invention should be interpreted in a broad sense, including but not limited to conventional replacement schemes not mentioned in this application, and also including direct implementation methods and indirect implementation methods.

[0020] Example 1 Combination Figure 1 and Figure 2 This embodiment describes a classification optimization method for intelligent question-answering multiple knowledge bases in the electric power industry, including the following steps: S1. Classify the power knowledge base according to the actual business division of the power enterprise to form multiple knowledge bases; S2, output the initial result through the large language model; S21. Query reception and classification: Receive the query input by the user and submit it to the large language model for classification; S12. Classification confidence distribution calculation: The large language model generates the confidence distribution of the classification of queries belonging to each knowledge base ,in The query belongs to the knowledge base The confidence level, is the number of knowledge bases; S23, classification results are determined: Determine the initial classification result according to the classification confidence , , that is, select the knowledge base with the highest confidence ; S24, initial answer generation: From the initial classification results Corresponding knowledge base Search Related content snippets , generating an initial answer through the large language model and save it.

[0021] S3, collect user feedback and trace the source of errors for negative feedback; S31. Obtain user feedback: If the user responds to the answer generated by the large language model to the query Feedback was collected and recorded, and all feedback was divided into positive feedback and negative feedback. Positive feedback is a like, indicating satisfaction. Negative feedback is a thumbs down, indicating dissatisfaction ; S32. Multi-source classification verification mechanism: For the recorded questions where users are not satisfied with the answers, it is necessary to determine whether they are caused by errors in the question classification process.

[0022] S321. Multiple knowledge base search answers: Send the query to all knowledge bases separately , retrieve content fragments, and generate a set of candidate answers ;in is the Nth candidate answer; S322, Answer Evaluation: Candidate answers Perform quality assessment and collect query and all knowledge bases Relevant content fragments are retrieved from the database, and the more powerful teacher model is called through the LLM-as-a-Judge method to select the best answer. , and its corresponding knowledge base is ; S323. Verification of classification results: If classification , then the classification is judged to be wrong, if , the classification is correct and the problem may lie in the retrieval or generation module S4. Build a case library and classify and manage cases; Case classification: Based on the multi-source classification verification results, the queries with user feedback are stored in the corresponding case library. The correctly classified ones are stored in the positive case library, and the incorrectly classified ones are stored in the negative case library. Each positive case record should include: query text; Initial classification results ; Classification confidence distribution ; Basis of classification; Timestamp.

[0023] The classification basis is generated by the teacher model: by collecting queries and retrieving relevant content fragments from all knowledge bases and , construct prompt words (Prompt) to guide the teacher model to generate classification-based reasoning.

[0024] Negative cases should include: query text; Initial classification results ; Classification confidence distribution ; The correct category determined by the multi-source classification verification mechanism ; Basis for correction; Timestamp.

[0025] The correction basis is generated by the teacher model: by collecting query, all knowledge base Relevant content fragments were retrieved from , and , construct a Prompt to guide the teacher model to generate error cause reasoning.

[0026] S5, calculating and ranking the weights of case queries; S51. Case weight calculation: The weight is calculated for each case query according to the following formula : (1) In the formula, the time factor Reflects the newness of the case: (2) Classification confidence factor :Among them, the case of , for negative cases .

[0027] Knowledge base coverage factor , reflecting the coverage of knowledge base types: (3) in, Representation Knowledge Base The number of times it is selected as the classification result in the current case set (positive and negative case libraries).

[0028] Information Entropy : Measuring classification uncertainty: (4) S52, Redundancy Detection and Optimization: Calculate the similarity between cases: (5) Set a threshold , if the similarity of a pair of cases is higher than , then delete the cases with lower weights.

[0029] S53. Case weight ranking: After removing redundancy, the positive and negative case sets are sorted according to their weights. S6. Design and generation of classification prompt word structure; S61. Summarize the guiding principles for classification; S62. Domain attribution analysis: Based on the functional division of each department in the power industry, business scenarios are matched with the knowledge base at a high level, and classification is guided by clarifying the core focus of the problem (such as customer service, power grid operation, equipment management, etc.).

[0030] S611, feature word extraction: using the correlation between high-frequency keywords and categories in historical data, through statistical analysis, we can mine feature word sets from different knowledge bases as a basis for rapid classification.

[0031] S612, scenario divergence resolution: For cross-category or boundary-blurred problems, by analyzing the commonalities and differences of easily confused scenarios, formulate clear distinction criteria and priority strategies to improve classification accuracy.

[0032] S613, Semantic Context Modeling: Combine context information to fully understand the background and purpose of the query, avoid misclassification caused by a single keyword, and ensure that the classification is consistent with the actual needs of the user.

[0033] S62, construct prompt words; The classification prompt words are divided into three parts: (1) Task description; Clarify the classification tasks and knowledge base selection rules.

[0034] (2) Presentation of positive and negative cases; Positive example template: Query: <query>; Classification result: <knowledge base name> (confidence: <classification confidence>); Classification basis: <reason for classification>; Negative example template: Query: <query> Error classification: <error knowledge base name> (confidence: <classification confidence>); Correct classification: <correct knowledge base name>; Correction basis: <Classification correction reason>; (3) Guiding principles for classification and correction S63, prompt word generation and deployment; S631, positive and negative case insertion rules: insert cases alternately according to the predetermined positive and negative case ratio (e.g. 3:1) to ensure that the classification rules are fully covered; S632, prompt word verification: evaluate the classification accuracy on the test set and adjust the prompt word content.

[0035] S633, prompt word online: Deploy the verified prompt words to the classification module to support real-time classification tasks.

[0036] Example 2 This embodiment is described in conjunction with Example 1. This embodiment discloses a classification optimization method for intelligent question-answering multiple knowledge bases in the power industry, which specifically includes the following steps: S1, knowledge base configuration; According to the actual business division of power enterprises, power knowledge can be divided into four major categories, and the corresponding knowledge bases are as follows: Electricity Marketing Regulatory Knowledge Base : Including electricity price policies, customer contract templates and other relevant regulations; Power Grid Dispatch System Knowledge Base :Including dispatching operation procedures, dispatching system standards, etc.; Power Asset Management Methods Knowledge Base : Including equipment management specifications, asset evaluation standards, etc.; Safety Supervision Method Knowledge Base : Including safety hazard detection methods, accident handling procedures, etc.

[0037] S2, the classification model outputs the initial results; S21, query reception and classification: Receive the query input by the user (specifically: "What are the provisions of the emergency plan?"), and submit it to the large language model (specifically: Qwen2.5-32B-Instruct) for classification processing; S22. Classification confidence distribution calculation: The model generates the classification confidence distribution of the query belonging to each knowledge base ,in The query belongs to the knowledge base The confidence level of the Qwen model is as follows: Electricity Marketing Regulatory Knowledge Base: =0.05; Power grid dispatching system knowledge base: =0.38; Power Asset Management Methods Knowledge Base: =0.55; Safety supervision disposal method knowledge base: =0.02; S23, classification result determination: determine the initial classification result based on the classification confidence , , that is, select the knowledge base with the highest confidence , here = , = .

[0038] S24. Initial answer generation: from classification results Corresponding knowledge base Retrieve relevant snippet content , generating initial responses through a large language model And save. For example, the retrieved fragments here are: :The emergency plan for dispatching accidents stipulates the use of a "red, orange, yellow and blue" four-color warning mechanism. Among them, the red warning requires XXX.

[0039] : People-oriented, safety first; unified command, hierarchical responsibility; rapid response, coordinated response; XXX.

[0040] : 1) Cut off the source of the accident in time to prevent the accident from expanding; 2) Organize emergency repair forces to carry out emergency repairs; XXX.

[0041] : Establish an emergency command platform and conduct emergency drills regularly, XXX.

[0042] The initial answer generated by the large language model is as follows: :The emergency plan stipulates the use of a "red, orange, yellow and blue" four-color warning mechanism. Among them, the red warning requires XXX.

[0043] S3, user feedback collection and error location; S31, user feedback acquisition: if the user is satisfied with the answer generated by the large language model to the query Feedback was given and the user's feedback was recorded. A thumbs-up indicates that the user is satisfied with the answer. , a thumbs down indicates that the user is not satisfied with the answer ; S32. Multi-source classification verification: For a given query and initial answer , it is necessary to determine whether the user’s click was caused by an error in the question classification process.

[0044] S321, multi-knowledge base search answer: send the query to all knowledge bases separately , retrieve content fragments, and generate a set of candidate answers ; : Sorry, I did not find any relevant regulations on emergency plans for dispatch accidents in the knowledge base. I suggest you try again with other queries.

[0045] : The emergency plan stipulates: It is necessary to contact the dispatch center immediately, confirm the scope of the accident, activate the emergency plan, and quickly restore power supply, XXX.

[0046] :The emergency plan stipulates the use of a "red, orange, yellow and blue" four-color warning mechanism. Among them, the red warning requires XXX.

[0047] : According to the "Power Dispatching Intelligence 2025 Plan", all dispatching accidents must first be analyzed by the AI ​​intelligent analysis system. The plan stipulates the establishment of a "cloud-edge-end" three-level response system XXX; Step 322: Answer evaluation: Perform quality assessment and collect query and knowledge base arrive Relevant content fragments are retrieved from the database, and the teacher model (Qwen-max can be used) is called through the LLM-as-a-Judge method to select the best answer , and its corresponding knowledge base is ; Step 323, MSCV result verification: LLM-as-a-Judge classification results show , indicating that the classification is incorrect and the negative case needs to be recorded.

[0048] Step 4: Case library construction and management; Case classification: According to the verification results of the MSCV process, for the questions that users are not satisfied with the answers to, determine whether they are caused by errors in the question classification process. The queries with user feedback are stored in the corresponding case library. The correctly classified ones are stored in the positive case library, and the incorrectly classified ones are stored in the negative case library. This case is a negative case and needs to be stored in the negative case library. The record is as follows: Query text: What are the provisions of the emergency plan? Initial classification results: , the knowledge base of power asset management methods; Classification confidence distribution: ; The correct category determined by the multi-source classification verification mechanism process: , power grid dispatching system knowledge base; Correction basis: Although Query mentioned "emergency plan", which is related to equipment repair in asset management, its core lies in "regulation". The system of power grid dispatching regulates the specific requirements of emergency plan, while the asset management method focuses more on the implementation of repair. Therefore, it should be classified into K2: Power Grid Dispatch System Knowledge Base.

[0049] Timestamp: Current time.

[0050] The MSCV (Multi-Source Classification Validation) is a classification error location process based on user feedback. When the user is dissatisfied with the answer, the system will trigger the MSCV process, generate a set of candidate answers by searching all knowledge bases in parallel, and verify the correctness of the initial classification results through the LLM-as-a-Judge method combined with the evaluation ability of the teacher model. Its core goal is to accurately locate classification errors or retrieval / generation module problems through multi-source answer comparison and quality assessment, and provide high-value negative samples for subsequent optimization.

[0051] Step 5: Case weight calculation and redundant annotation; Step 51: Case weight calculation: Calculate the weight for this case query according to the following formula:

[0052] in: Time Factor : Cases generated on the day; Classification confidence factor ; Knowledge base coverage factor (the frequency of the power grid dispatch system knowledge base is 7): ; Information Entropy: 0.9245; Total weight: ; Step 52, redundancy detection and optimization: Calculate the similarity between this case and other cases, and set the threshold value in the present invention. , then if the similarity , remove cases with lower weights.

[0053] Step 53, case weight sorting: sort the positive and negative case sets after deleting redundancy according to weight.

[0054] Step 6: Design and generate classification prompt word structure; Step 61: Summarize the classification guidelines; Step 611, domain attribution analysis: divide the responsibilities of each business field, clarify the core concerns of each field, map these concerns to knowledge base categories, and form the "core business field distinction" rules.

[0055] Step 612, feature word extraction: by analyzing historical classification data, extract high-frequency keywords with significant differentiation in different knowledge bases, and organize them into category keyword lists to form "keyword feature recognition" rules.

[0056] Step 613, scenario divergence resolution: For easily confused scenarios in the classification history, the system summarizes the reasons for the blurred classification boundaries, and formulates a judgment strategy based on the priority of focus points in combination with business logic and actual needs, forming a "principle for handling easily confused scenarios"; Step 614, semantic context modeling: Utilize the context and query background, combined with a complete semantic understanding model or rule, to clarify the real business needs of classification, ensure the accuracy of classification decisions, and form a "context comprehensive judgment" rule.

[0057] Step 62, constructing prompt words; Classification prompt word structure design; 1. Task Description system: #Role: User Intent Identification Assistant ##Profile -language: Chinese -description: Helps users determine the type of knowledge base to which the question belongs, so as to guide queries to relevant knowledge bases for different questions.

[0058] ##Knowledge Electricity marketing regulations knowledge base: including electricity price policies, customer contract templates and other related regulations; Power grid dispatching system knowledge base: including dispatching operation procedures, dispatching system standards, etc.; Power asset management method knowledge base: including equipment management specifications, asset evaluation standards, etc.; Safety supervision disposal method knowledge base: including safety hazard investigation methods, accident handling procedures, etc.

[0059] Skills (1) Able to identify different types of users’ questions.

[0060] (2) Based on the question content, accurately match the most relevant knowledge base.

[0061] (3) When it is impossible to make an accurate judgment, provide the option of "unable to determine clear intent".

[0062] ##Goals Assist users to automatically match the appropriate knowledge base when querying to optimize the efficiency and relevance of question answering.

[0063] ##Rules (1) Quickly filter and match the most suitable knowledge base based on the keywords and topics of the question.

[0064] (2) When the question involves multiple fields or the keywords are vague, the prompt "Unable to determine the clear intention" will be displayed.

[0065] (3) Avoid excessive speculation. If the question is not clearly expressed or the keywords do not match, no further matching will be performed.

[0066] 2. Positive and negative case presentation Positive example: Query: What are the inspection requirements and cycles for 35kV substation equipment? Classification results: Power asset management method knowledge base (confidence: 0.92); Classification basis: This question involves the inspection management specifications of substation equipment and belongs to the category of equipment management. The query content is directly related to the inspection system and cycle standards in the equipment management specifications, which is a typical asset management question.

[0067] Negative example: Query: What is the on-site handling procedure after an electric shock accident? Error classification: Power grid dispatch system knowledge base (confidence: 0.68); Correct classification: safety supervision disposal method knowledge base; Correction basis: Although the accident may have occurred on power grid equipment, this is a typical safety accident handling problem. Such emergency rescue and on-site handling plans should be queried in the safety supervision handling method knowledge base, especially in the accident handling procedures section. The incorrect classification may be because the keyword "accident" has a certain correlation with dispatch accident handling, but based on the complete context, this is obviously a safety accident handling problem.

[0068] 3. Classification Guiding Principles (1) Differentiation of core business areas Marketing knowledge base: all issues related to customer service, electricity charges, metering, contracts and other business relationships; Dispatching knowledge base: all issues related to grid operation, load control, power balance and other system regulation; Asset management knowledge base: all issues related to physical management such as equipment maintenance, repair procedures, asset evaluation, etc. Safety supervision knowledge base: all safety management issues related to personal safety, accident handling, hidden danger investigation, etc. (2) Keyword feature identification Marketing: electricity prices, billing, contracts, users, charges, subsidies, etc.; Dispatching: load, regulation, dispatch, frequency, voltage, etc.; Assets: equipment, overhaul, maintenance, inspection, evaluation, etc.; Safety supervision: safety, accidents, hidden dangers, protection, emergency response, etc.; (3) Principles for handling confusing scenes Equipment-related issues: It is necessary to distinguish whether it is asset management or safety hazard; If the focus is on maintenance → asset management; If the focus is on safety risks → safety supervision and disposal; Accident-related issues: It is necessary to distinguish whether it is a dispatching accident or a safety accident; If it involves system operation → dispatch system; If personal safety is involved → Safety supervision disposal; Inspection-related issues: It is necessary to distinguish whether it is an equipment inspection or a safety inspection; If the focus is on equipment status → asset management; If the focus is safety control → safety supervision disposal; (4) Comprehensive judgment based on context Don’t rely on just a single keyword; Need to be combined with the complete problem context; Consider the query purpose and application scenario; Prioritize the knowledge base of the business responsible department; Step 63, generating and deploying prompt words; Step 631, positive and negative case insertion rules: insert cases alternately at a positive and negative case ratio of 3:1 to ensure that the classification rules are fully covered; Step 632, prompt word verification: Use the test set to verify that the classification accuracy reaches 97%, passing the online test; Step 633, prompt words online: deploy the generated prompt words to the classification module and enable real-time classification service.

[0069] Example 3 This embodiment is explained in conjunction with Example 1. This embodiment discloses a multi-knowledge base classification optimization system for intelligent question answering in the power industry, including a knowledge base classification module, a large language model, a feedback processing module, a classification management module, a weight calculation module and a classification prompt word construction module; The knowledge base classification module is used to classify the power knowledge base according to the actual business division of the power enterprise to form multiple knowledge bases; The large language model is used to answer the query and output an initial result; The feedback processing module is used to collect user feedback and perform error tracing and location for negative feedback; The classification management module is used to build a case library and classify and manage cases; The weight calculation module is used to calculate and sort the weights of case queries; The classification prompt word construction module is used to design and generate the classification prompt word structure.< / query> < / query>

Claims

1. A classification optimization method for intelligent question-answering multi-knowledge bases in the power industry, characterized in that: The steps include: S1. Classify the power knowledge base according to the actual business division of the power enterprise to form multiple knowledge bases; S2, output the initial result through the large language model; S3, collect user feedback and trace the source of errors for negative feedback; S4. Build a case library and classify and manage cases; According to the multi-source classification verification results, the queries with user feedback are stored in the corresponding case library; S5, calculating and ranking the weights of case queries; S6. Design and generate classification prompt word structures to guide the large language model to classify and optimize the user's query.

2. According to claim 1, a classification optimization method for intelligent question-answering multiple knowledge bases in the power industry is characterized by: In S2, the initial result includes an initial classification result and an initial answer; the specific steps of outputting the initial result through the large language model are as follows: S21. Query reception and classification: Receive the query input by the user and submit it to the large language model for classification; S22. Classification confidence distribution calculation: The large language model generates the confidence level of the query in each knowledge base. distributed ,in The query belongs to the knowledge base Confidence level; is the number of knowledge bases; S23, classification results are determined: Determine the initial classification result according to the classification confidence , , that is, select the knowledge base with the highest confidence ; S24, initial answer generation: From the initial classification results Corresponding knowledge base Search Related content snippets , generating an initial answer through the large language model and save it.

3. According to claim 2, a classification optimization method for intelligent question-answering multiple knowledge bases in the power industry is characterized in that: In S3, the specific steps of collecting user feedback and tracing the source of errors for negative feedback are as follows: S31. Obtain user feedback: Get the user's initial answer to the Feedback, and divide all feedback into positive feedback and negative feedback; S32. Multi-source classification verification mechanism: For the query corresponding to the negative feedback and the initial answer , trace the root cause of the negative feedback, that is, whether the negative feedback is caused by errors in the problem classification link.

4. According to claim 3, a classification optimization method for intelligent question-answering multiple knowledge bases in the power industry is characterized in that: In S32, tracing the root cause of the negative feedback specifically includes the following steps: S321. Multiple knowledge base search answers: Send the query to all knowledge bases separately , retrieve content fragments, and generate a set of candidate answers ; S322, Answer Evaluation: For the candidate answer set Perform quality assessment and collect query and all knowledge bases Retrieve relevant content snippets from the search results and select the best answer , and its corresponding knowledge base is ; S323. Verification of classification results: If classification , then the classification is judged to be wrong, if , the classification is correct.

5. According to claim 1, a classification optimization method for intelligent question-answering multiple knowledge bases in the power industry is characterized in that: In S5, the specific steps of calculating and ranking the weights of the case queries are as follows: S51. Case weight calculation: Calculate weights for case queries , the weight The calculation formula is: （1） in, is the time factor, is the classification confidence factor, is the knowledge base coverage factor, is the information entropy, is the total weight; Refers to the time factor coefficient, Refers to the classification confidence factor coefficient, refers to the knowledge base coverage factor coefficient, refers to information entropy, and ; S52, Redundancy Detection and Optimization: Calculate the similarity between the case and other cases. If the similarity is greater than the preset threshold, delete the cases whose weight is lower than the preset threshold. S53. Case weight ranking: After removing redundancy, the positive and negative case sets are sorted according to their weights.

6. The method for optimizing classification of multiple knowledge bases for intelligent question answering in the electric power industry according to claim 1 is characterized in that: In S6, the specific steps of designing and generating the classification prompt word structure are as follows: S61, preset classification guidelines; S62, construct prompt words; Design the structure of the classification prompt words according to the task description, positive and negative case presentations, and classification guidelines; S63. Prompt word generation and deployment.

7. The method for optimizing classification of multiple knowledge bases for intelligent question answering in the electric power industry according to claim 6 is characterized in that: In S61, the preset classification guideline specifically includes the following steps: S611, Domain Attribution Analysis: Divide the responsibilities of each business domain, clarify the core focus of each domain, map the core focus to the knowledge base category, and form the "core business domain distinction" rule; S612, feature word extraction: by analyzing historical classification data, extracting keywords from different knowledge bases according to the degree of discrimination, and organizing them into category keyword lists to form "keyword feature recognition" rules; S613, scenario divergence resolution: for scenarios that are confusing in the classification history, summarize the reasons for the fuzzy classification boundaries, and formulate a judgment strategy based on the priority of focus points in combination with business logic and actual needs to form a "principle for handling easily confusing scenarios"; S614, semantic context modeling: use the context and query background, combined with a complete semantic understanding model or rules, to clarify the real business needs of classification, ensure the accuracy of classification decisions, and form a "context comprehensive judgment" rule.

8. The method for optimizing classification of multiple knowledge bases for intelligent question answering in the electric power industry according to claim 6 is characterized in that: In S63, the prompt word generation and deployment specifically includes the following steps: S631, positive and negative case insertion rules: insert cases alternately according to the preset positive and negative case ratio to ensure that the classification rules are fully covered; S632, prompt word verification: Use the test set to verify the classification accuracy. If the classification accuracy meets the standard, the online test is passed; S633, prompt words online: deploy the generated prompt words to the classification module and enable real-time classification service.

9. A classification optimization system based on the classification optimization method of intelligent question-answering multi-knowledge base in the power industry according to any one of claims 1 to 8, characterized in that: It includes a knowledge base classification module, a large language model, a feedback processing module, a classification management module, a weight calculation module and a classification prompt word construction module; The knowledge base classification module is used to classify the power knowledge base according to the actual business division of the power enterprise to form multiple knowledge bases; The large language model is used to answer the query and output an initial result; The feedback processing module is used to collect user feedback and perform error tracing and location for negative feedback; The classification management module is used to build a case library and classify and manage cases; The weight calculation module is used to calculate and sort the weights of case queries; The classification prompt word construction module is used to design and generate the classification prompt word structure.

Images