Model question and answer optimization method and device, computer equipment and storage medium

Abstract

The embodiment of the invention relates to a model question and answer optimization method and device, computer equipment and a storage medium. A data extraction mode of each knowledge data document is determined according to a document type, data extraction operation is performed on each knowledge data document, and extracted data is stored in a database; when a target problem is received, performing problem intention classification on the target problem to obtain a problem type; splitting the target question into at least one sub-question according to the question type; determining a target database corresponding to each sub-question, and retrieving candidate answers corresponding to the sub-questions; and combining the sub-questions and the candidate answers, and inputting the combined sub-questions and candidate answers into a large model to generate Therefore, differential data extraction can be performed on multiple types of operation and maintenance documents, the documents can be stored in different libraries according to the document types, question intention classification, targeted sub-question splitting and accurate retrieval in different libraries are combined, the retrieval accuracy and answer generation efficiency of network operation and maintenance knowledge questions and answers are improved, and the quick response requirement of complex operation and maintenance questions is met.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of large language model technology, and in particular to a model question answering optimization method, apparatus, computer device and storage medium. Background Technology

[0002] With the rapid development of mobile internet and the continuous expansion of network base station scale, the coexistence of multiple standards and manufacturers' equipment has led to a surge in the number of network operation and maintenance (O&M) related documents, covering various types such as equipment manuals, O&M manuals, historical work orders, and spreadsheet-based technical documents. These documents contain a massive amount of O&M knowledge, but their complex formats and scattered data make it difficult for front-end O&M personnel to quickly and accurately obtain the information they need. Many repetitive issues need to be transferred to back-end experts for processing, which wastes expert resources and is not conducive to front-end personnel accumulating experience. The efficiency and knowledge reuse rate of network O&M urgently need to be improved.

[0003] To address the challenges of retrieving and answering massive amounts of professional network operations and maintenance (O&M) data, existing technologies often employ a Retrieval-Augmented Generation (RAG) process. This involves segmenting documents using fixed-length or general rules, constructing a single vector database, vectorizing user questions, performing similarity searches, and finally generating answers based on the search results. However, this data processing approach is rigid. Using a uniform standard to segment different data types, such as multi-level title documents, standardized work order tables, and professional comparison tables, easily disrupts semantic logic, leading to lost data connections and semantic breaks in the text. Furthermore, the problem handling lacks specificity, failing to categorize and break down network O&M issues based on their professional attributes and solution processes. This makes it difficult to accurately match the complex question-and-answer needs of vertical domains. Simultaneously, a single database is prone to data confounding, failing to meet the core requirement of efficient question-and-answering in O&M scenarios.

[0004] Therefore, existing technologies suffer from low retrieval accuracy and insufficient targeted answer generation in the field of network operations and maintenance (O&M) Q&A, making it difficult to quickly respond to the demand for answers to complex O&M problems. Thus, improving the accuracy and efficiency of knowledge-based Q&A in network O&M scenarios has become an urgent technical problem to be solved. Summary of the Invention

[0005] In view of this, in order to solve the above-mentioned technical problems or some of the technical problems, the present invention provides a model question answering optimization method, apparatus, computer equipment and storage medium.

[0006] In a first aspect, embodiments of the present invention provide a model question-answering optimization method, comprising: The data extraction method for each knowledge data document is determined based on its document type; According to the data extraction method, data extraction operations are performed on each of the knowledge data documents, and the extracted data is stored in the corresponding database according to the document type. When a target question is received, the target question is classified according to its intent to obtain the question type. The target problem is divided into at least one sub-problem based on the problem type; Determine the target database corresponding to each sub-problem, and retrieve the candidate answers corresponding to the sub-problem from the target database; After combining at least one of the sub-questions and their corresponding candidate answers, the combination is input into a large model so that the large model generates the target answer corresponding to the target question.

[0007] Secondly, embodiments of the present invention provide a model question-answering optimization device, comprising: The determination module is used to determine the data extraction method for each knowledge data document based on the document type of each knowledge data document; The processing module is used to perform data extraction operations on each of the knowledge data documents according to the data extraction method, and store the extracted data into the corresponding database according to the document type. The question classification module is used to classify the question intent when a target question is received, and to obtain the question type of the target question; A splitting module is used to split the target problem into at least one sub-problem based on the problem type; A retrieval module is used to determine the target database corresponding to each sub-question and retrieve the candidate answers corresponding to the sub-question from the target database. The generation module is used to combine at least one of the sub-questions and their corresponding candidate answers, and then input them into a large model so that the large model generates the target answer corresponding to the target question.

[0008] Thirdly, embodiments of the present invention provide a computer device, including: a processor and a memory, wherein the processor is configured to execute a model question answering optimization program stored in the memory to implement the model question answering optimization method described in any one of the first aspects above.

[0009] Fourthly, embodiments of the present invention provide a storage medium storing one or more programs, which can be executed by one or more processors to implement the model question answering optimization method described in any of the first aspects above.

[0010] The model question-answering optimization method provided in this invention determines the data extraction method for each knowledge data document based on its document type; performs data extraction operations on each knowledge data document according to the extraction method, and stores the extracted data into corresponding databases according to document type; when a target question is received, the target question is classified according to its intent to obtain the question type; the target question is split into at least one sub-question according to the question type; a target database corresponding to each sub-question is determined, and candidate answers corresponding to the sub-questions are retrieved from the target database; the at least one sub-question and its corresponding candidate answers are combined and input into a large model so that the large model generates the target answer corresponding to the target question. Therefore, by using differentiated data extraction and storing data in separate databases according to document type for multiple types of operation and maintenance documents, combined with question intent classification, targeted sub-question splitting, and precise database retrieval, the retrieval accuracy and answer generation efficiency of network operation and maintenance knowledge question answering are significantly improved, meeting the rapid response requirements for complex operation and maintenance problems. Attached Figure Description

[0011] Figure 1 A flowchart illustrating a model question-answering optimization method provided in an embodiment of the present invention; Figure 2 A flowchart illustrating another model question-answering optimization method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a model question answering optimization device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0014] Figure 1 This is a flowchart illustrating a model question-answering optimization method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method specifically includes: S11. Determine the data extraction method for each knowledge data document based on its document type; S12. Perform data extraction operations on each knowledge data document according to the data extraction method, and store the extracted data into the corresponding database according to the document type.

[0015] The model-based question-and-answer optimization method provided in this invention is applied to computer equipment, including but not limited to servers and desktop computers. The application scenario is network operation and maintenance knowledge question-and-answer, specifically applicable to the operation and maintenance work of operators, base station equipment manufacturers, and other entities. When front-end operation and maintenance personnel, technical support personnel, or related training personnel encounter network operation and maintenance related issues such as base station equipment fault diagnosis, parameter configuration queries, historical case references, and interference level judgment, they can raise target questions through natural language. The system, based on massive operation and maintenance documents (equipment manuals, historical work orders, technical tables, etc.), through precise retrieval and large-scale model generation, quickly outputs professional and targeted answers. By using differentiated data extraction for various types of operation and maintenance documents and storing them in separate databases according to document type, combined with question intent classification and targeted sub-question splitting, and precise database retrieval, the retrieval accuracy and answer generation efficiency of network operation and maintenance knowledge question-and-answer are significantly improved.

[0016] In this embodiment, knowledge data documents refer to various documents in the network operation and maintenance field that carry professional knowledge. These may include: user manuals, operation and maintenance manuals, historical operation and maintenance work order records, indicator comparison tables, etc., provided by equipment manufacturers, and serve as the original data source for data extraction. Document type characterization represents the categories divided according to the format characteristics and content attributes of knowledge data documents. Document types may include: explanatory documents with multi-level headings, historical record documents with standardized formats, tabular data documents carrying rules / comparison relationships, etc., and serve as the classification basis for differentiated data extraction.

[0017] Data extraction methods represent the use of different information extraction logic and operational processes for different document types. The core objective is to transform unstructured / semi-structured documents into structured, semantically complete data suitable for subsequent retrieval and storage in a database. Data extraction methods may include: constructing question-and-answer pairs by associating titles, rewriting work order tables into structured text, and reconstructing linear text from table row and column relationships.

[0018] Specifically, the document types are first divided according to the format and content characteristics of each knowledge data document. Then, based on the attribute differences of each type of knowledge data document, appropriate tools and preset extraction logic are used to extract data, ensuring that the extracted data is structured, semantically complete, and suitable for subsequent database storage.

[0019] Pre-set corresponding data extraction methods for different document types to extract and store content from various knowledge data documents. First, perform document type pre-classification. Based on the format characteristics and content attributes of knowledge data documents, divide them into three categories: The first category is explanatory documents with multi-level headings, such as equipment user manuals, technical specifications, and maintenance manuals. The core characteristic is the presence of clear first-, second-, and third-level headings, and the format is often DOCX or PDF. The second category is historical record documents, such as maintenance work orders. The core characteristic is that they are presented in a fixed table format, with preset fields such as "phenomenon description, cause analysis, troubleshooting process, and solution." These are mostly maintenance work order records in PDF format with standardized formatting. The third category is tabular data documents (e.g., indicator comparison tables). The core characteristic is that they contain complete table titles, row titles, and column titles. The table content carries network operation and maintenance related rules (such as indicator comparisons, parameter standards, etc.), and the format can be a PDF with embedded tables or an Excel spreadsheet. Then, obtain the corresponding data extraction method for each document type.

[0020] In one possible implementation, the data extraction method for each knowledge data document is determined based on its document type, including: In the case of a descriptive document containing multi-level headings, the data extraction method for the knowledge data document is determined as follows: extract the multi-level headings from the knowledge data document, construct question-answer pairs based on the hierarchical relationship between the multi-level headings and the main text content, and use these as the extracted data to store in the first database; When the document type is a historical document, the data extraction method for the knowledge data document is determined as follows: the work order table in the knowledge data document is read as a whole according to the preset field order, and the content of each field is rewritten as continuous text as the extracted data for storage in the second database. The work order table is the record carrier of operation and maintenance event information, and the operation and maintenance event information includes at least the following fields: phenomenon of operation and maintenance event, cause analysis, troubleshooting process, and solution. When the document type is a tabular data document, the data extraction method for the knowledge data document is determined as follows: for the operation and maintenance rule table in the knowledge data document, the text obtained by semantically splitting and reconstructing the table content according to the correspondence between the row header and column header is used as the extracted data for storage in a third database. The operation and maintenance rule table is a structured carrier that carries network operation and maintenance related rules.

[0021] In this embodiment, for explanatory documents containing multi-level headings, an Optical Character Recognition (OCR) tool, fine-tuned with data from the network operations and maintenance field, is used to scan the document. This tool identifies text segments that are on separate lines and contain line breaks (\n). Headings are then categorized into first-level, second-level, and third-level headings according to the document's layout logic, forming a hierarchical heading list. Associations are constructed following the rule that the upper-level heading is the question, and the lower-level headings and their corresponding text are the answers: the first-level heading is the core question, and all its subordinate second-level headings together constitute the answer; the second-level heading is a sub-question, and its subordinate third-level headings or directly corresponding text content serve as the answer; for second-level headings without subordinate headings, their subsequent text content is directly associated as the question answer, thus constructing question-answer pairs. The association results are organized into standardized question-answer pairs of "Q - question text, A - answer text," ensuring that each question-answer pair is semantically independent and fully covered. These pairs are directly stored in the first database, which stores the data extracted from the explanatory documents.

[0022] As an example: A document's primary heading is "SA Access Issues," with secondary headings such as "SIM Card Not Read" and "No Signaling," creating a question-and-answer pair: "Q: What are some SA access issues? A: These include not reading the SIM card, no signaling, and failed random access process." The secondary heading "SIM Card Not Read" is linked to its body text, creating a question-and-answer pair: "Q: What are the solutions for the 'SIM card not read' issue in SA access issues? A: XXX (body text corresponding to the secondary heading)."

[0023] For historical documents, structured text extraction from work order tables is required. OCR tools are used to recognize the table structure in PDF work orders, accurately extracting the text content corresponding to preset fields such as "phenomenon," "cause analysis," "troubleshooting process," and "solution," avoiding field confusion or missing information. During extraction, the table data is parsed using code, strictly following the preset field order (e.g., "phenomenon-cause analysis-troubleshooting process-solution"), sequentially reading fragmented information from each field (e.g., test data in the phenomenon field, parameter relationships in the cause analysis field, and step details in the troubleshooting process field). This fragmented information is then integrated into logically coherent structured text, preserving the complete processing chain of the maintenance event and avoiding semantic breaks. The final structured text is stored in a second database, which is used to store the data extracted from the historical documents.

[0024] As an example: Extract data from different fields in the work order table for "low data rate problem caused by power saturation" and rewrite it as follows: "Phenomenon description: During the test, the power level, SINR, etc. of a certain section are normal, but the low MCS causes poor uplink data rate; Cause analysis: The low MCS is caused by factors such as inner and outer loop processes and parameter configurations. Specifically, it can be investigated from aspects such as equipment fault alarms, interference, parameter configuration, and uplink / downlink switching; Troubleshooting process: 1. The background network management system tracks alarms and confirms that there are no alarms at the site during the test; 2. The background spectrum scan shows that the NI index is normal; Solution: Check that the parameter RssiPowerReduceSwitch is false, modify this parameter and its supporting parameters RssipowerReduceThr, etc., and then retest. The problem is resolved." Simultaneously, a question-and-answer pair can be generated: "Q: Phenomenon description of low data rate problem caused by power saturation; A: During the test, the power level, SINR, path loss, and packet loss of a certain section are all normal, but the low MCS causes poor uplink data rate."

[0025] Linear text extraction based on row and column relationships is performed on tabular data documents. OCR tools are used to fully extract table titles, row titles, column titles, and the content of each cell, clarifying the logical relationship of "table topic - row dimension conditions - column dimension standards - corresponding results." Following the logic of "row title content + column title conditions + corresponding results," each row of table data is semantically decomposed, transforming the structured table data into linear text expressed in natural language, while retaining the table title as a text prefix to ensure semantic integrity. All reconstructed linear text is integrated to form a text set covering all table rules, and stored in a third database. This third database is used to store the data extracted from the tabular data document.

[0026] As an example: Consider the following table showing the correspondence between noise and interference levels in residential areas:

[0027] After extracting the table title and row / column content, it is reconstructed into linear text: "When the average cell noise is less than -116dBm, interference level: no interference; when the average cell noise is greater than -116dBm and less than or equal to -110dBm, interference level: slight interference; when the average cell noise is greater than -110dBm and less than or equal to -100dBm, interference level: moderate interference; when the average cell noise is greater than -110dBm, interference level: strong interference."

[0028] Furthermore, after determining the data extraction methods for different document types, data extraction operations are performed on the knowledge data documents of each document type according to the data extraction methods, and the extracted data is stored in the database corresponding to the document type.

[0029] S13. When a target question is received, the target question is classified according to its intent to obtain the target question type.

[0030] In this embodiment, after receiving the target question, it is input into a question classification model trained based on the BERT architecture. This model has been trained and optimized through five types of question samples in the field of network operation and maintenance. It can analyze the expression characteristics, semantic orientation and solution process requirements of the question, identify the corresponding question attributes, and then accurately classify the target question into one of the five preset question types.

[0031] Five question types can be preset: 1. First document content questions (requiring comprehensive answers from multiple dimensions): These are network operation and maintenance related questions rewritten from the first-level headings of explanatory documents, requiring comprehensive answers from multiple dimensions. 2. Historical work order questions (requiring the search for similar operation and maintenance cases and solutions): These are questions extracted from historical work order tables using OCR tools and rewritten, requiring the search for similar operation and maintenance cases and solutions. 3. Second document content questions (no rewriting required, answers can be directly retrieved): These are questions rewritten from a combination of first- and second-level headings of explanatory documents, requiring no multi-dimensional analysis and allowing direct retrieval of answers. 4. Follow-up questions (no clear subject, requires context): These are questions without a clear subject, requiring context from the previous round of question and answer to clarify their meaning, and are sub-questions generated by the large model. 5. Logical Reasoning Problems (requiring rule retrieval and data comparison reasoning): These are problems without direct answers that require two steps of reasoning. The first step is to retrieve relevant knowledge, and the second step is to extract relevant data based on the retrieval results for further simple comparison before the problem can be solved. This requires first extracting the data from the table using a trained OCR tool, creating some question templates based on the table data, and then rewriting the question templates and table data using a large model to create more questions.

[0032] For five question types, a question classification model is pre-trained. Before training the model, training data corresponding to each of the five question types is obtained to form a training dataset, including: Extract the first-level headings from the explanatory documents, and rewrite the first-level headings into question-style text using a large model to obtain the first document content question set; The work order form text in the historical documents is extracted using an OCR tool. The work order form text is then divided based on preset keywords. The title of each work order form text is then extracted. The titles of the work order form text are then rewritten into question-style text using a large model to obtain a set of historical work order questions. Extract the second-level headings from the explanatory documents, combine them with the first-level headings, and rewrite them into question-style text using a large model to obtain the second document content question set. In this case, if there are no third-level headings under the second-level headings, the body text corresponding to the second-level headings is determined as the answer; if there are third-level headings under the second-level headings, the body text corresponding to the third-level headings is determined as the answer. Create subjectless follow-up question samples, and rewrite the follow-up question samples through similar semantic expansion using a large model to generate text in question form, thus obtaining a set of follow-up questions. Follow-up question samples need to be combined with the previous question to generate the final question. OCR tools are used to extract the operation and maintenance rule table information from the tabular data document, and a problem template is created. The problem template and operation and maintenance rule table information are then rewritten into text in the form of a problem requiring two-step reasoning through a large model, resulting in a set of logical reasoning problems. The training dataset is generated from the first document content question set, the historical work order question set, the second document content question set, the follow-up question set, and the logical reasoning question set.

[0033] In this embodiment, explanatory documents containing multi-level headings (e.g., equipment operation and maintenance manuals, technical specifications) are used as the data source. An OCR tool, finely tuned for network operation and maintenance, accurately identifies first-level headings that are on separate lines and include line breaks, forming a list of first-level headings. A suitable prompt template is constructed, containing the message: "Rewrite the following technical topics in the network operation and maintenance field into professional questions requiring comprehensive multi-dimensional answers: {first-level heading}". Each first-level heading is embedded in a template placeholder and input into a large language model in batches. Based on the professional expression logic of the operation and maintenance field, the model rewrites the headings into abstract, multi-dimensional, complex questions (e.g., rewriting "RF module outage fault" as "What are the causes of RF module outage faults?", and rewriting "SA access problems" as "What are the core manifestations and handling ideas of SA access-related problems?"). The questions generated by the model are filtered, eliminating samples with unclear expressions and weak relevance to the topic, retaining those that meet the characteristic of "requiring multi-dimensional breakdown and answering", and integrating them to form the first document content question set.

[0034] When constructing the historical work order issue set, historical records (standardized maintenance work orders) in PDF format are used as the data source. A dedicated OCR tool is used to identify the table structure within the work orders, accurately extracting the text content corresponding to preset fields such as "phenomenon description, cause analysis, troubleshooting process, and solution." Based on preset keywords (e.g., problem name, phenomenon overview, fault description), the extracted work order table text is segmented, and the core theme title of each work order is located (e.g., "Probabilistic communication anomalies of terminals near XX base station," "Low rate problem caused by power saturation"). One core title is extracted for each work order. A Prompt template is designed: "Rewrite the core theme of the following maintenance work orders into a professional question requiring the search for similar historical cases and solutions: {Work Order Title}". The extracted work order title is embedded in the template and input into a large language model. The model rewrites the title into a specific question pointing to the case query (e.g., rewriting "Probabilistic communication anomalies of terminals near XX base station" as "How to handle probabilistic communication anomalies of terminals near XX base station?"). Verify the professionalism and rationality of the generated questions, ensuring that each question focuses on historical case references and solution query needs, and integrate them to form a historical work order question set.

[0035] For the construction of the second document content question set, explanatory documents continue to be used as the data source. OCR tools are used to extract first-level headings and their subordinate second-level headings. These are then logically combined into semantically complete phrases (e.g., a first-level heading of "SA Access Issues" and a subordinate second-level heading of "SIM card not read," resulting in "SA Access Issues: SIM card not read"). The document content corresponding to the second-level headings is retrieved simultaneously. If there is no third-level heading under the second-level heading, the subsequent text content of the second-level heading is marked as the corresponding answer for the combined phrase. If there is a third-level heading under the second-level heading, the corresponding text content of the third-level heading is summarized as the corresponding answer for the combined phrase, ensuring a clear relationship between questions and answers. A Prompt template is constructed: "Rewrite the following technical phrases into professional questions that do not require multi-dimensional decomposition and can be directly searched for answers: {first-level + second-level heading combined phrases}". These combined phrases are embedded into the template and input into the large model to generate concise and clear query questions (e.g., rewriting "SA Access Issues: SIM card not read" as "What are the solutions for SA Access Issues: SIM card not read?"). Ensure that each question meets the characteristic of "answers can be obtained directly by searching", bind the corresponding answer tags (for subsequent model training annotation), and integrate them to form a second document content question set.

[0036] To construct the follow-up question set, subjectless initial samples of follow-up questions are created by manually simulating network operation and maintenance Q&A scenarios. The core feature is that "the semantics need to be understood in the context of the previous round of Q&A," such as "What are the solutions to these problems?" "What is the corresponding troubleshooting process?" "What is the scope of impact?" etc. The number of initial samples needs to cover the main operation and maintenance consultation scenarios. The Prompt template is designed as follows: "Based on the Q&A scenario in the field of network operation and maintenance, the follow-up questions '{initial follow-up question samples}' of the previous round of questions '{simulated previous round of questions}' are semantically expanded to generate more follow-up questions with consistent style, no subject, and contextual relevance, requiring professional expression and logical coherence." Among them, "simulated previous round of questions" selects typical questions from the question sets of the first / second document content (such as "What are the SA access problems?"), which are combined with the initial follow-up question samples and input into the large model. The extended samples generated by the filter model are filtered out, and questions with semantic repetition, logical confusion or explicit subjects are removed. Valid samples that are subjectless, context-dependent, and consistent with the operation and maintenance scenario are retained and integrated to form a set of follow-up questions. Each sample is then bound with a corresponding simulated question label from the previous round to clarify the contextual relationship.

[0037] For constructing a set of logical reasoning questions, it is necessary to rewrite the large model into smaller questions of different levels according to the progressive logical relationship of reasoning. Using tabular data documents as the data source, OCR tools are used to accurately extract table titles, row titles, column titles, and cell content (e.g., "Correspondence Table of Cell Noise and Interference Levels" and "Correspondence Table of Parameter Configuration and Performance Indicators"), fully preserving the logical relationship between "conditions and results" (e.g., "Average cell noise <-116dBm" is the condition, and "No interference" is the result). Two-step reasoning question templates are manually created based on the table logic. The core structure is known conditions + reasoning query, such as "The noise detection value of XXX cell is XXdBm (known condition), does it cause interference? (reasoning query)" or "A base station parameter XXX is set to XX (known condition), does the corresponding performance indicator meet the standard? (reasoning query)". The templates must cover the core logical relationships of the tabular data. Construct a Prompt template: "Combining the following operation and maintenance rule table information: {extracted structured table content}, rewrite the problem template '{template}' into a specific logical reasoning problem according to the format of 'known conditions + reasoning query'. The first step requires retrieving rule conditions, and the second step requires deriving conclusions based on condition comparison." Combine the extracted table information (converted into structured text) with the problem template and input it into the large model so that the large model generates a problem that meets the characteristics of two-step reasoning (e.g., generating 'XXX cell noise detection value is -112dBm, does it cause interference?' based on the 'Cell Noise and Interference Level Correspondence Table'), bind the corresponding table information tags, and integrate them to form a set of logical reasoning problems.

[0038] The first document content question set, the historical work order question set, the second document content question set, the follow-up question set, and the logical reasoning question set constructed above are summarized, and a unique category label is assigned to the samples in each question set (e.g., the first document content question label is 0, the historical work order question label is 1, etc.). At the same time, a structured annotation of question text-category label-corresponding context / answer label is added, and finally a training dataset with uniform format, comprehensive coverage, and clear annotation is formed, which is used for subsequent training of question classification models based on the BERT architecture.

[0039] After constructing the training dataset, train the problem classification model using the following method: For training data of different question types in the training dataset, the training data is combined with the question description templates corresponding to the question types to form prompt words in a preset format; Input the prompt words into the large model, and the large model learns the problem characteristics and expression logic of the training data to generate new training data that conforms to the characteristics of different problem types. Add the new training data to the training dataset to expand the scale. An initial classification model is built based on the BERT architecture. The expanded training dataset is input into the initial classification model. By adjusting the parameters of the initial classification model, the initial classification model learns the attribute features and classification boundaries of different problems until the classification accuracy of the initial classification model meets the preset threshold, thus obtaining the trained problem classification model.

[0040] In this embodiment, based on the question-and-answer process requirements of network operation and maintenance scenarios, a description template tailored to the core characteristics of each question type is designed. The template includes at least: question type definition, description logic constraints, and professional scenario limitations, ensuring that the large model is guided to generate questions that conform to the type attributes. The specific template is as follows: The first document content question: The template is "This question belongs to the complex question in the field of network operation and maintenance that requires a comprehensive answer from multiple dimensions. It needs to generate a professional question that conforms to the logic of 'problem description - accompanying alarm - scope of impact - handling steps' around the core scenarios such as fault handling and technical explanation. Refer to the training data features: {first document content question set}"; Historical work order issues (case reference required): The template is "This issue belongs to the network operation and maintenance historical work order query category. It is necessary to focus on the complete processing link of the operation and maintenance event and generate a professional question that conforms to the structure of 'phenomenon description-problem analysis-troubleshooting process-solution'. Refer to the training data characteristics: {historical work order issue set}"; The second document content question: The template is "This question belongs to the direct query type of network operation and maintenance questions. It does not require multi-dimensional breakdown. Generate a concise and clear professional question that can be directly matched with the document answer. Refer to the training data features: {second document content question set}"; Follow-up questions: The template is "This question belongs to the subjectless follow-up question in the network operation and maintenance Q&A scenario. It needs to rely on the context of the previous round of Q&A to clarify the semantics and generate a professional follow-up question of the same level as the previous question. Refer to the training data features: {follow-up question set}"; Logical reasoning problem: The template is "This problem belongs to the category of network operation and maintenance logical reasoning problem. There is no direct answer. It needs to generate a two-step reasoning problem by following the progressive logic of 'premise query - data comparison - conclusion deduction'. Refer to the training data characteristics: {Logical Reasoning Problem Set}".

[0041] For the training data of five different problem types in the training dataset, the text content of a single training data is embedded into the corresponding template's "{...}" placeholder, supplemented with network operation and maintenance domain professional terminology constraints (such as: base station, MCS, interference level, work order, etc.), forming standardized prompt words of "type definition + logical constraints + professional limitation + training data reference", ensuring that the prompt words can accurately guide the large model to learn the target problem features.

[0042] The prompts, categorized by question type, are batch-input into a large language model (such as a general-purpose large model based on the Transformer architecture). The model learns the core attributes of each question type (such as the multi-dimensional decomposition logic of complex problems, the subjectless nature of follow-up questions, and the progressive relationships of reasoning questions) and the professional expression style of network operations and maintenance scenarios by analyzing the type constraints, expression logic, and professional features of the training data within the prompts. Based on the learned features, the large model generates multiple new training samples of the same type for each input prompt (the number can be preset, such as 5-10 samples per prompt). Then, a combination of rule-based filtering and manual sampling is used to select effective samples, eliminating invalid samples that are semantically repetitive, logically incoherent, irrelevant to the operations and maintenance scenario, or do not conform to the characteristics of the corresponding question type. The selected effective new samples are then merged with the original training data to expand the training dataset.

[0043] An initial classification model was built based on the BERT architecture. Basic model parameters adapted to the network operations and maintenance domain (such as hidden layer dimension, number of attention heads, number of layers, etc.) were configured, and model weights pre-trained on general Chinese corpora were loaded to reduce the difficulty of training convergence for the domain model. The expanded training dataset was divided into training and validation sets according to a preset ratio (e.g., 8:2). All samples underwent text preprocessing, including word segmentation using a network operations and maintenance-specific vocabulary, converting the text into vector codes recognizable by the model, and assigning unique category labels to the five types of questions (e.g., 0 for complex document content, 1 for historical work orders, etc.), forming structured training data of "text vector - category label". The preprocessed training set is input into the initial classification model. Using the cross-entropy loss function as the optimization objective, the model parameters (such as learning rate, regularization coefficient, batch size, etc.) are iteratively adjusted through backpropagation to allow the model to continuously learn the attribute features and classification boundaries of various problems. During training, the model's classification accuracy is evaluated using a validation set after each training round. If the accuracy does not reach a preset threshold (e.g., 90%), the process returns to the second step to expand the training samples for the corresponding low-accuracy category, or adjusts the model parameters (e.g., optimizing the learning rate decay strategy, increasing the number of training rounds) until the model's classification accuracy on the validation set stably meets the preset threshold. Once the model's classification accuracy reaches the preset requirement, training stops, and the final model weights, parameter configurations, and domain-specific vocabulary are saved, resulting in the trained problem classification model. This model can be directly used for subsequent type recognition tasks after receiving the target problem. The training of this problem classification model is now complete.

[0044] Furthermore, upon receiving a target question, the target question is categorized by its intent to determine its question type, including: The target question is input into the question classification model to identify the expression characteristics, semantic orientation and solution requirements of the target question, thereby obtaining the question intent corresponding to the target question, and determining the question type of the target question based on the question intent.

[0045] In this embodiment, the problem classification model has a pre-built fixed mapping rule of "problem intent - problem type", where: the intent of "requiring a comprehensive answer from multiple dimensions" corresponds to the first document content problem; the intent of "requiring the search for similar operation and maintenance cases and solutions" corresponds to the historical work order problem; the intent of "no need to break down, can directly match the answer" corresponds to the second document content problem; the intent of "no clear subject, needs to be associated with the previous context" corresponds to the follow-up question problem; and the intent of "no direct answer, needs to be searched and reasoned step by step" corresponds to the logical reasoning problem.

[0046] The system receives network operation and maintenance related target questions input by users. The questions are in the form of natural language text. The target question text is standardized, including removing redundant spaces and special characters, unifying the expression of professional terminology, and ensuring that the text format is consistent with the expression style of the model training samples to avoid format differences affecting recognition accuracy. The preprocessed target questions are input into the trained question classification model. The model extracts the expression features of the questions (such as whether there is no subject, whether it contains inference keywords such as "whether" or "how to compare") and semantic orientation (such as the core theme being fault handling, case query, direct query, etc.) through a multi-layer attention mechanism. It also combines the domain knowledge learned during training to analyze the core solution requirements of the questions (such as whether multi-dimensional decomposition is required, whether contextual association is required, and whether step-by-step reasoning is required). Question intent matching: The model compares and matches the obtained "expression features, semantic orientation, and solution requirements" with the built-in intent feature library of five question types to determine the core intent corresponding to the target question.

[0047] For example: if the question contains expressions such as "what happened" or "what are the processing steps", the semantics point to a multi-dimensional solution to the problem, and the solution requirement is a comprehensive breakdown. In this case, it matches the intent of "requiring a comprehensive solution from multiple dimensions", which corresponds to the first document content question. If the question has no subject (such as "what are the solutions to these problems?"), the semantics point to dependence on the preceding text, and the solution requirement is to relate to the context. In this case, it matches the intent of "requiring association with the previous context", which corresponds to a follow-up question. If the question contains inference expressions such as "whether" or "comparison", the semantics point to rule verification, and the solution requirement is step-by-step retrieval and deduction. In this case, it matches the intent of "requiring step-by-step retrieval and reasoning", which corresponds to a logical reasoning question.

[0048] Based on the matched question intent, the model accurately classifies the target question into one of five question types according to the preset mapping rules, and outputs the identifier of the question type (e.g., labels 0-4 correspond to the five question types respectively).

[0049] S14. Break down the target problem into at least one sub-problem based on the problem type.

[0050] In this embodiment, based on the five types of target questions, corresponding Prompt templates are matched (e.g., complex questions are adapted to the "multi-dimensional decomposition template", follow-up questions are adapted to the "context splicing template", and logical reasoning questions are adapted to the "progressive reasoning template"). The large model rewrites different target questions into at least one professional sub-question based on the Prompt. Questions that do not need to be rewritten are not split. Complex questions and historical work order questions are split into parallel sub-questions. Follow-up questions are spliced ​​with the previous round of dialogue and then split into sub-questions of the same level. Logical reasoning questions are split into multi-level sub-questions according to progressive logic.

[0051] In one possible implementation, the target problem is broken down into at least one sub-problem based on the problem type, including: When the problem type is a first document content problem or a historical work order problem, the target problem is broken down into multiple parallel sub-problems using a large model; When the problem type is a second document content problem, the target problem is treated as a sub-problem; When the question type is a follow-up question, the target question is concatenated with the question and answer text from the previous round of question and answer. The concatenated target question is then split into multiple parallel sub-questions using a large model. When the problem type is a logical reasoning problem, the target problem is broken down into multiple sub-problems at different levels through a large model. Each level of sub-problem is used to concatenate the retrieval results of the previous level of sub-problem in the target database.

[0052] In this embodiment, if the problem type is a first document content problem or a historical work order problem, the appropriate Prompt template is invoked (e.g., the first document content problem template focuses on "fault description - accompanying alarm - scope of impact - handling steps", and the historical work order problem template focuses on "phenomenon description - problem analysis - troubleshooting process - solution"). The target problem is input into the pre-trained large model. The large model splits the target problem into multiple semantically independent and dimensionally complete parallel sub-problems according to the template logic, ensuring that each sub-problem accurately corresponds to the retrieval needs of a core type of information. The parallel sub-problems need to be retrieved from the corresponding target database at the same time.

[0053] If the target question type is a second document content question, there is no need to split it further. The original target question can be directly used as a unique sub-question, retaining its concise and clear search attributes, and adapting to the need to directly retrieve answers from the target database.

[0054] If the target question type is a follow-up question, the complete text of the previous question-and-answer process (including the previous question and the corresponding answer) is first extracted from the system session cache. This text is then concatenated with the current target question to form a semantically coherent complete text. This text is then input into the large model and the "context-related splitting template" is called. The large model defines the semantic direction based on the concatenated text and splits it into multiple parallel sub-questions corresponding to the subdivided scenarios of the previous answer. The parallel sub-questions need to be retrieved from the corresponding target database at the same time.

[0055] If the target question type is a logical reasoning question, the "two-step reasoning splitting template" is called. The target question is input into the large model, and the model is split into multiple sub-questions of different levels according to the progressive logic of "rule query - data comparison - conclusion deduction". The sub-question of the next level reserves an interface for splicing the search results of the previous level sub-question, which provides logical support for subsequent progressive search. When searching, the sub-questions of different levels need to be searched in the target database in the order of their levels.

[0056] For example, the target question is "The noise detection value in XX community is -112 dBm. Does it cause interference?" First, call the "Two-step Reasoning and Splitting Prompt Template" (template content: "This question needs to be split according to the logic of 'rule query - data comparison - conclusion derivation'. In the first step, retrieve relevant rule conditions, and in the second step, derive a conclusion based on the conditions for comparison."), input the target question into the large model. The model first splits it into the first-level sub-question "What is the corresponding threshold rule for community noise and interference level?" (rule query), and then splits it into the second-level sub-questions "What is the magnitude relationship between -112 dBm and the interference level threshold? (data comparison) Does it meet the interference determination conditions? (conclusion derivation)". Among them, the second-level sub-questions reserve an interface for splicing the retrieval results of the first-level sub-questions. Subsequently, it can combine with the rules such as "When the average community noise < -116 dBm, there is no interference; when -116 dBm < X ≤ -110 dBm, there is slight interference" retrieved from the target database for the first-level sub-question to complete progressive reasoning.

[0057] All the split sub-questions retain the association identifier with the original question type to ensure that the subsequent sub-library model can accurately match the database corresponding to the sub-question and achieve targeted retrieval.

[0058] As an example, taking the following table as an example, for each of the above five question types, at least one sub-question is split respectively, including question examples generated by different question types, the Prompt template used when the question examples are rewritten by the large model, and the sub-questions obtained after being rewritten by the large model:

[0059] S15. Determine the target database corresponding to each sub-question and retrieve the candidate answers corresponding to the sub-question in the target database.

[0060] In this embodiment, input each of the split sub-questions into the sub-library model trained based on the BERT architecture. This model determines the target database corresponding to each sub-question according to the mapping rules between the question types of different sub-questions and the database preset; subsequently, in the corresponding target database, retrieve the information highly relevant to the sub-question through semantic similarity matching, and screen out the results with qualified accuracy as the candidate answers for each sub-question.

[0061] In a possible implementation manner, determining the target database corresponding to each sub-question and retrieving the candidate answers corresponding to the sub-question in the target database includes: Sub-problems are input into the database sharding model so that the model outputs the target database corresponding to the sub-problems. The database sharding model is trained based on the BERT architecture and pre-associates the mapping relationship between problem types and databases. The mapping relationship includes: the correspondence between the first document content problem, the second document content problem, and follow-up questions and the first database; the correspondence between historical work order problems and the second database; and the correspondence between logical reasoning problems and the third database. When the sub-question is a question about the content of the first document, a question about historical work orders, or a follow-up question, multiple parallel sub-questions are searched simultaneously in the corresponding target database, and multiple search results are used as candidate answers. When the sub-question is a question about the content of a second document, the corresponding target database is searched for the sub-question, and the search results are used as candidate answers. When the sub-problem is a logical reasoning problem, the first-level sub-problem is searched in the corresponding target database. The search results for the first-level sub-problem, the first-level sub-problem and the second-level sub-problem are concatenated. The concatenated content is then searched in the target database again until no lower-level sub-problem exists. The current search result is then used as the candidate answer.

[0062] In this embodiment, all the sub-questions after being split are input one by one into a database-splitting model trained based on the BERT architecture. This model is configured with a preset mapping relationship between question types and databases, including: the sub-questions corresponding to the first document content question, the second document content question, and follow-up questions are all mapped to the first database (a question-and-answer pair database of explanatory documents containing multi-level headings); the sub-questions corresponding to the historical work order question are mapped to the second database (a structured text database of historical work orders); and the sub-questions corresponding to the logical reasoning question are mapped to the third database (a linear text database of tables). The model directly outputs the target database identifier for each sub-question by identifying the original question type associated with the sub-question, thus clarifying the search scope.

[0063] If the sub-problem is a first document content problem, a historical work order problem, or a follow-up question (all of which are multiple parallel sub-problems), then based on the target database identifier output by the database sharding model, a retrieval request is simultaneously initiated to the corresponding database. During the retrieval, a semantic similarity matching algorithm is used to recall information in the database that is highly semantically related to each sub-problem. Each parallel sub-problem obtains retrieval results independently, and all results together constitute a candidate answer set for this type of problem, ensuring coverage of core information in all dimensions.

[0064] If the sub-question is a question about the content of the second document, then a search is initiated directly in the first database that it is mapped to. The unique or optimal associated result corresponding to the sub-question is extracted through precise semantic matching, and the result is used as a candidate answer.

[0065] If the sub-problem is a logical reasoning problem, the process follows a hierarchical retrieval and result concatenation logic: First, for the first-level sub-problem, the corresponding rule-based results (such as interference level threshold rules) are retrieved from the third database; then, the first-level sub-problem, its retrieval results, and the second-level sub-problem are concatenated into a semantically coherent retrieval statement, which is then retrieved again from the third database, using the preceding rule results to constrain the retrieval scope; if there are more levels of sub-problems, the above concatenation and retrieval process is repeated until the last level of sub-problems is processed, and the final retrieval result is used as the candidate answer for the logical reasoning problem.

[0066] The entire process uses a database sharding model to pinpoint the precise search range, combined with differentiated search strategies for sub-question types. This ensures both efficient parallel recall of parallel sub-questions and progressive, precise derivation of logical reasoning questions, guaranteeing that candidate answers closely match the needs of the sub-questions.

[0067] S16. Combine at least one sub-problem and its corresponding candidate answer, and input them into the large model so that the large model can generate the target answer corresponding to the target problem.

[0068] In this embodiment, if the original target question is a first document content question or a historical work order question, then all parallel sub-questions corresponding to the target question and their corresponding candidate answers are concatenated in the order of "sub-question 1 + candidate answer 1 + sub-question 2 + candidate answer 2 + ..." to form a complete combination text of question and answer association, ensuring that the core dimensions of each sub-question (such as fault description, processing steps, phenomenon analysis, etc.) and the corresponding search results are not omitted, and preserving the integrity of multi-dimensional information.

[0069] If the target question is a question about the content of the second document: directly concatenate the individual sub-question and its candidate answer into a combined text of sub-question and candidate answer, without additional expansion, highlighting the core features of direct retrieval and accurate matching.

[0070] If the target question is a follow-up question, the previous round of question and answer text (including the previous round of questions and corresponding answers) is first extracted from the system session cache. Then, the previous round of question and answer text is concatenated with the combined text of the current parallel sub-questions and candidate answers to obtain the combined text, forming a semantically coherent complete information chain, clarifying the logical connection between the follow-up question and the preceding text.

[0071] If the target question is a logical reasoning question, it is concatenated in a progressive order of "first-level sub-question + candidate answer + second-level sub-question + candidate answer" to obtain a combined text, which fully preserves the reasoning chain and ensures that the basis for the solution of the later-level sub-question is clearly related to the search results of the earlier level.

[0072] Design customized Prompt templates for different question types and network operation and maintenance scenarios, clearly define the requirements for generating large models, and avoid answers that deviate from the professional context: The first document content question and historical work order question template is as follows: "Based on the following sub-questions in the field of network operation and maintenance and the corresponding search results, a comprehensive and systematic summary is generated to produce a professional answer to the target question. It should cover the core dimensions of each sub-question, be logically coherent, use accurate technical terms, and meet the practical needs of operation and maintenance scenarios: {combined text}". The second document content question template is: "Based on the following sub-questions and the combined text of the search results, generate a concise and accurate professional answer to the target question on network operation and maintenance, without the need for additional expansion or supplementation: {combined text}"; Follow-up question template: "Combining the context of the previous round of questions and answers with the following sub-questions and search results, generate a professional answer that echoes the previous round's answer and is highly targeted, ensuring semantic coherence and focusing on the core of the follow-up question: {combined text}"; Logical reasoning question template: "Based on the following progressive sub-questions and the combined text of search results, generate rigorous network operation and maintenance reasoning answers according to the logic of "rule retrieval - data comparison - conclusion deduction", clearly define the deduction process and final conclusion, and provide clear technical basis: {combined text}".

[0073] The combined text corresponding to the question type is embedded into template placeholders to form standardized input content, ensuring a clear generation direction for the large model. The filled standardized Prompt is then input into the large model, which has been adapted and optimized using network operations and maintenance data. Based on template constraints and the association information of the "combined text," the large model integrates semantics, organizes logic, and generates a target answer that meets the following requirements: Professionalism: Accurately uses network operations and maintenance terminology; answers are tailored to practical scenarios such as equipment maintenance and troubleshooting. Logical Structure: Complex questions are explained hierarchically by sub-question dimension; reasoning questions clearly present the progressive derivation process; follow-up questions echo the preceding text without disconnection. Completeness: Covers the core information of all sub-questions, with no key search results omitted. Conciseness: Outputs accurate answers directly without rewriting the questions, avoiding redundant expansion. Ultimately, the large model outputs structured and professional target answers, directly feeding back to the user and meeting the precise question-and-answer needs of network operations and maintenance scenarios.

[0074] The model question-answering optimization method provided in this invention aims to solve the problems of inaccurate retrieval and the tendency of large models to produce illusions in network operation and maintenance scenarios. Its core solution includes three parts: First, data preprocessing and multi-database construction. For three types of network operation and maintenance data—explanatory documents, historical records, and tabular data documents—a differentiated extraction method is used. For explanatory documents containing multi-level headings, a self-trained OCR tool is used to extract the headings and construct question-answer pairs. For formatted historical work order documents, OCR extracts the tabular text and rewrites it into structured text. For tabular data documents, the row and column relationships are reconstructed into linear text. The three types of processed data are stored separately in databases to achieve database-based storage and avoid data confusion. Second, question classification and professional decomposition. A classification model is trained based on the BERT architecture to classify user questions... Based on the answer process attributes, questions are categorized into five types: complex document content questions, historical work order questions, questions where document content does not require rewriting, follow-up questions, and logical reasoning questions. Then, using an adapted Prompt template, a large model is invoked to break down complex questions into multiple parallel sub-questions. Follow-up questions are broken down by incorporating the context of the previous round of question-and-answer sessions, and logical reasoning questions are broken down into multi-level progressive sub-questions. Thirdly, database sharding and answer generation are performed. A BERT-based database sharding model matches each sub-question to its corresponding database, retrieves them using different methods, and finally combines the sub-questions with corresponding candidate answers. This data is then input into a large model optimized with data from the operations and maintenance domain to generate professional and accurate final answers. This approach solves the retrieval bias problems caused by the traditional RAG technology's inability to uniformly process multiple data types and its lack of targeted question breakdown, significantly improving the accuracy and professionalism of network operations and maintenance knowledge Q&A.

[0075] Figure 2 A flowchart illustrating another model question-answering optimization method provided in an embodiment of the present invention is shown below. Figure 2 The method specifically includes: Data preprocessing and database construction: The first step involves constructing three dedicated databases for three types of documents in the network operations and maintenance field (including explanatory documents with multi-level headings, historical record documents with standardized formats, and tabular data documents containing rules) using differentiated data extraction methods: Multi-level headings are extracted from explanatory documents and question-and-answer pairs are constructed and stored in the first database; the work order tables in historical record documents are rewritten into structured text and stored in the second database; and the tabular data documents are reconstructed into linear text according to row and column relationships, forming the third database, ensuring that data storage is accurate and adaptable to subsequent retrieval needs.

[0076] User issue handling and multi-path execution: The second step is to classify user issues by intent using a BERT-based classification model to determine the issue type (issues where document content does not need to be rewritten (second document content issues), issues with complex document content (first document content issues), historical work order issues, follow-up questions, and logical reasoning issues). Then, the subsequent decomposition, retrieval, and answer generation are performed according to the following classification paths: The first category (questions for which the document content does not need to be rewritten): No splitting is required. The original question is directly input into the database sharding model and matched to the first database. The answer is obtained through precise semantic retrieval. The second category (complex document content issues and historical work order issues): First, the corresponding professional Prompt template is assembled and input into the large model to break down the original problem into multiple parallel sub-problems; then, the target database is matched for each sub-problem through the database-splitting model (complex document content issues correspond to the first database, and historical work order issues correspond to the second database), and candidate answers for each sub-problem are obtained through parallel retrieval; finally, the "sub-problems + corresponding candidate answers" are combined in logical order and input into the large model to summarize and generate a comprehensive and professional final answer; The third type (follow-up questions): First, extract the complete question and answer text from the previous round from the session cache, and concatenate it with the current follow-up question to form a semantically coherent complete text; then, following the process of the second type of questions, after concatenating the Prompt template, split it into multiple parallel sub-questions through the large model, match the candidate answers in the first database through the database-splitting model, and finally integrate them by the large model to generate a targeted answer that echoes the previous text; The fourth category (logic reasoning problems): First, a dedicated Prompt template is assembled and input into the main model to break down the original problem into multi-level progressive sub-problems. The first step involves inputting the first-level sub-problem into the sub-database model to match it with a third database, retrieving rule-based candidate answers. The second step involves concatenating "first-level sub-problem + its candidate answer + second-level sub-problem" into a complete search text, and then retrieving deduction-based candidate answers from the third database. Finally, the two levels of sub-problems and their corresponding candidate answers are combined and input into the main model to generate a rigorous reasoning answer according to the logic of "rule retrieval - data comparison - conclusion deduction".

[0077] Figure 3 This is a schematic diagram of the structure of a model question-answering optimization device provided in an embodiment of the present invention, as shown below. Figure 3 The device specifically includes: The determining module 31 is used to determine the data extraction method for each knowledge data document based on the document type of each knowledge data document; Processing module 32 is used to perform data extraction operations on each of the knowledge data documents according to the data extraction method, and store the extracted data into the corresponding database according to the document type; The question classification module 33 is used to classify the question intent of the target question when a target question is received, and to obtain the question type of the target question; The splitting module 34 is used to split the target problem into at least one sub-problem according to the problem type; The retrieval module 35 is used to determine the target database corresponding to each sub-question and retrieve the candidate answers corresponding to the sub-question from the target database; The generation module 36 is used to combine at least one of the sub-questions and their corresponding candidate answers, and then input them into a large model so that the large model generates the target answer corresponding to the target question.

[0078] In one possible implementation, the determining module is specifically used to determine, when the document type is an explanatory document containing multi-level headings, the data extraction method for the knowledge data document is: extracting multi-level headings from the knowledge data document, constructing question-answer pairs based on the hierarchical relationship between the multi-level headings and the text content, and using these as the extracted data for storage in the first database; When the document type is a historical document, the data extraction method for the knowledge data document is determined as follows: the work order table in the knowledge data document is read as a whole according to the preset field order, and the content of each field is rewritten as continuous text as the extracted data for storage in the second database. The work order table is a record carrier of operation and maintenance event information, and the operation and maintenance event information includes at least the following fields: phenomenon of operation and maintenance event, cause analysis, troubleshooting process, and solution. When the document type is a tabular data document, the data extraction method for the knowledge data document is determined as follows: for the operation and maintenance rules table in the knowledge data document, the text obtained by semantically splitting and reconstructing the table content according to the correspondence between row headers and column headers is used as the extracted data for storage in a third database. The operation and maintenance rules table is a structured carrier that carries network operation and maintenance related rules.

[0079] In one possible implementation, the question classification module is further configured to combine the training data of different question types in the training dataset with the question expression templates corresponding to the question types to form prompt words in a preset format. The prompt words are input into a large model, which learns the problem features and expression logic of the training data to generate new training data that conforms to the characteristics of different problem types. The new training data is then added to the training dataset to expand its scale. An initial classification model is built based on the BERT architecture. The expanded training dataset is input into the initial classification model. By adjusting the parameters of the initial classification model, the initial classification model learns the attribute features and classification boundaries of different problems until the classification accuracy of the initial classification model meets the preset threshold, thus obtaining a trained problem classification model.

[0080] In one possible implementation, the question classification module is further used to extract the first-level headings from the explanatory documents, and rewrite the first-level headings into question-form text using a large model to obtain a first document content question set; The work order form text in the historical document is extracted using an OCR tool. The work order form text is then divided based on preset keywords. The title of each work order form text is extracted and rewritten into question form using a large model to obtain a historical work order question set. Extract the second-level headings from the explanatory documents, combine them with the first-level headings, and rewrite them into question-style text using a large model to obtain the second document content question set. Where there are no third-level headings under the second-level headings, the body text corresponding to the second-level headings is determined as the answer; when there are third-level headings under the second-level headings, the body text corresponding to the third-level headings is determined as the answer. Create subjectless follow-up question samples, and rewrite the follow-up question samples using a large model to generate text in question form, thus obtaining a set of follow-up questions. The follow-up question samples need to be combined with the previous question to generate the final question. The operation and maintenance rule table information in the table data document is extracted by OCR tool, a question template is created, and the question template and the operation and maintenance rule table information are rewritten into text in the form of questions requiring two-step reasoning through a large model to obtain a set of logical reasoning questions. The training dataset is generated by combining the first document content question set, the historical work order question set, the second document content question set, the follow-up question set, and the logical reasoning question set.

[0081] In one possible implementation, the problem classification module is specifically used to input the target problem into the problem classification model, so as to identify the expression features, semantic orientation and answer requirements of the target problem through the problem classification model, obtain the problem intent corresponding to the target problem, and determine the problem type of the target problem based on the problem intent.

[0082] In one possible implementation, the splitting module is specifically used to split the target problem into multiple parallel sub-problems using a large model when the problem type is the first document content problem or the historical work order problem; If the problem type is the second document content problem, the target problem is taken as the sub-problem; When the question type is the follow-up question, the target question is concatenated with the question and answer text from the previous round of question and answer, and the concatenated target question is split into multiple parallel sub-questions using a large model; When the problem type is the logical reasoning problem, the target problem is broken down into multiple sub-problems at different levels using a large model. Each sub-problem at each level is used to concatenate the retrieval results of the previous level's sub-problem in the target database.

[0083] In one possible implementation, the retrieval module is specifically used to input the sub-question into the database sharding model, so that the database sharding model outputs the target database corresponding to the sub-question. The database sharding model is trained based on the BERT architecture and pre-associates the mapping relationship between question types and databases. The mapping relationship includes: the correspondence between the first document content question, the second document content question, and the follow-up question and the first database; the correspondence between the historical work order question and the second database; and the correspondence between the logical reasoning question and the third database. When the sub-question is the first document content question, the historical work order question, or the follow-up question, multiple parallel sub-questions are simultaneously searched in the corresponding target database, and multiple search results are used as candidate answers. If the sub-question is the second document content question, a search is performed in the corresponding target database for the sub-question, and the search results are used as candidate answers; When the sub-problem is the logical reasoning problem, the corresponding target database is searched for the first-level sub-problem. The search results corresponding to the first-level sub-problem, the first-level sub-problem and the second-level sub-problem are concatenated. The concatenated content is then searched again in the target database until no next-level sub-problem exists. The current search result is then used as the candidate answer.

[0084] The model question-answering optimization device provided in this embodiment can be as follows: Figure 3 The apparatus shown can perform, as Figure 1 All steps of the optimization method for the model problem are then implemented to achieve... Figure 1 For details on the technical effectiveness of the optimization method for the model problem shown, please refer to [link / reference]. Figure 1 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0085] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 4The computer device 400 shown includes at least one processor 401, a memory 402, at least one network interface 404, and other user interfaces 403. The various components in the computer device 400 are coupled together via a bus system 405. It is understood that the bus system 405 is used to implement communication between these components. In addition to a data bus, the bus system 405 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 4 The general designated all buses as Bus System 405.

[0086] The user interface 403 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0087] It is understood that the memory 402 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 402 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0088] In some implementations, memory 402 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 4021 and application program 4022.

[0089] The operating system 4021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 4022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 4022.

[0090] In this embodiment of the invention, by calling the program or instructions stored in the memory 402, specifically the program or instructions stored in the application program 4022, the processor 401 executes the method steps provided in each method embodiment, including, for example: The data extraction method for each knowledge data document is determined based on its document type; According to the data extraction method, data extraction operations are performed on each of the knowledge data documents, and the extracted data is stored in the corresponding database according to the document type. When a target question is received, the target question is classified according to its intent to obtain the question type. The target problem is divided into at least one sub-problem based on the problem type; Determine the target database corresponding to each sub-problem, and retrieve the candidate answers corresponding to the sub-problem from the target database; After combining at least one of the sub-questions and their corresponding candidate answers, the combination is input into a large model so that the large model generates the target answer corresponding to the target question.

[0091] The methods disclosed in the above embodiments of the present invention can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in the form of software. The processor 401 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 402. Processor 401 reads the information in memory 402 and, in conjunction with its hardware, completes the steps of the above method.

[0092] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0093] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0094] The computer device provided in this embodiment may be as follows: Figure 4 The device shown can perform, for example Figure 1 All steps of the optimization method for the model problem are then implemented to achieve... Figure 1 For details on the technical effectiveness of the optimization method for the model problem shown, please refer to [link / reference]. Figure 1 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0095] This invention also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; it may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and it may also include combinations of the above types of memory.

[0096] When one or more programs in the storage medium can be executed by one or more processors to implement the above-described model question-answering optimization method executed on the device side.

[0097] The processor is used to execute a model question-answering optimization program stored in memory to implement the following steps of the model question-answering optimization method executed on the device side: The data extraction method for each knowledge data document is determined based on its document type; According to the data extraction method, data extraction operations are performed on each of the knowledge data documents, and the extracted data is stored in the corresponding database according to the document type. When a target question is received, the target question is classified according to its intent to obtain the question type. The target problem is divided into at least one sub-problem based on the problem type; Determine the target database corresponding to each sub-problem, and retrieve the candidate answers corresponding to the sub-problem from the target database; After combining at least one of the sub-questions and their corresponding candidate answers, the combination is input into a large model so that the large model generates the target answer corresponding to the target question.

[0098] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0099] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0100] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A model question-answering optimization method, characterized in that, include: The data extraction method for each knowledge data document is determined based on its document type; According to the data extraction method, data extraction operations are performed on each of the knowledge data documents, and the extracted data is stored in the corresponding database according to the document type. When a target question is received, the target question is classified according to its intent to obtain the question type. The target problem is divided into at least one sub-problem based on the problem type; Determine the target database corresponding to each sub-problem, and retrieve the candidate answers corresponding to the sub-problem from the target database; After combining at least one of the sub-questions and their corresponding candidate answers, the combination is input into a large model so that the large model generates the target answer corresponding to the target question.

2. The method according to claim 1, characterized in that, The step of determining the data extraction method for each knowledge data document based on its document type includes: When the document type is a descriptive document containing multi-level headings, the data extraction method for the knowledge data document is determined as follows: extract the multi-level headings from the knowledge data document, construct question-answer pairs based on the hierarchical relationship between the multi-level headings and the main text content, and use them as the extracted data for storage in the first database; When the document type is a historical document, the data extraction method for the knowledge data document is determined as follows: the work order table in the knowledge data document is read as a whole according to the preset field order, and the content of each field is rewritten as continuous text as the extracted data for storage in the second database. The work order table is a record carrier of operation and maintenance event information, and the operation and maintenance event information includes at least the following fields: phenomenon of operation and maintenance event, cause analysis, troubleshooting process, and solution. When the document type is a tabular data document, the data extraction method for the knowledge data document is determined as follows: for the operation and maintenance rules table in the knowledge data document, the text obtained by semantically splitting and reconstructing the table content according to the correspondence between row headers and column headers is used as the extracted data for storage in a third database. The operation and maintenance rules table is a structured carrier that carries network operation and maintenance related rules.

3. The method according to claim 2, characterized in that, Before receiving the target question, the method further includes: For training data of different question types in the training dataset, the training data is combined with the question expression template corresponding to the question type to form prompt words in a preset format; The prompt words are input into a large model, which learns the problem features and expression logic of the training data to generate new training data that conforms to the characteristics of different problem types. The new training data is then added to the training dataset to expand its scale. An initial classification model is built based on the BERT architecture. The expanded training dataset is input into the initial classification model. By adjusting the parameters of the initial classification model, the initial classification model learns the attribute features and classification boundaries of different problems until the classification accuracy of the initial classification model meets the preset threshold, thus obtaining a trained problem classification model.

4. The method according to claim 3, characterized in that, The question types include: questions about the content of the first document, questions about historical work orders, questions about the content of the second document, follow-up questions, and logical reasoning questions; The training dataset was generated in the following way: Extract the first-level headings from the explanatory documents, and rewrite the first-level headings into question-style text using a large model to obtain the first document content question set; The work order form text in the historical document is extracted using an OCR tool. The work order form text is then divided based on preset keywords. The title of each work order form text is extracted and rewritten into question form using a large model to obtain a historical work order question set. Extract the second-level headings from the explanatory documents, combine them with the first-level headings, and rewrite them into question-style text using a large model to obtain the second document content question set. Where there are no third-level headings under the second-level headings, the body text corresponding to the second-level headings is determined as the answer; when there are third-level headings under the second-level headings, the body text corresponding to the third-level headings is determined as the answer. Create subjectless follow-up question samples, and rewrite the follow-up question samples using a large model to generate text in question form, thus obtaining a set of follow-up questions. The follow-up question samples need to be combined with the previous question to generate the final question. The operation and maintenance rule table information in the table data document is extracted by OCR tool, a question template is created, and the question template and the operation and maintenance rule table information are rewritten into text in the form of questions requiring two-step reasoning through a large model to obtain a set of logical reasoning questions. The training dataset is generated by combining the first document content question set, the historical work order question set, the second document content question set, the follow-up question set, and the logical reasoning question set.

5. The method according to claim 3, characterized in that, When a target question is received, the target question is classified according to its intent to obtain the question type, including: The target question is input into the question classification model to identify the expression features, semantic orientation, and answer requirements of the target question through the question classification model, thereby obtaining the question intent corresponding to the target question, and determining the question type of the target question based on the question intent.

6. The method according to claim 4, characterized in that, The step of breaking down the target problem into at least one sub-problem according to the problem type includes: When the problem type is either the first document content problem or the historical work order problem, the target problem is split into multiple parallel sub-problems using a large model; If the problem type is the second document content problem, the target problem is taken as the sub-problem; When the question type is the follow-up question, the target question is concatenated with the question and answer text from the previous round of question and answer, and the concatenated target question is split into multiple parallel sub-questions using a large model; When the problem type is the logical reasoning problem, the target problem is broken down into multiple sub-problems at different levels using a large model. Each sub-problem at each level is used to concatenate the retrieval results of the previous level's sub-problem in the target database.

7. The method according to claim 6, characterized in that, The step of determining the target database corresponding to each sub-question and retrieving candidate answers corresponding to the sub-questions from the target database includes: The sub-problem is input into the database sharding model so that the database sharding model outputs the target database corresponding to the sub-problem. The database sharding model is trained based on the BERT architecture and pre-associates the mapping relationship between the problem type and the database. The mapping relationship includes: the correspondence between the first document content problem, the second document content problem and the follow-up question and the first database; the correspondence between the historical work order problem and the second database; and the correspondence between the logical reasoning problem and the third database. When the sub-question is the first document content question, the historical work order question, or the follow-up question, multiple parallel sub-questions are simultaneously searched in the corresponding target database, and multiple search results are used as candidate answers. If the sub-question is the second document content question, a search is performed in the corresponding target database for the sub-question, and the search results are used as candidate answers; When the sub-problem is the logical reasoning problem, the corresponding target database is searched for the first-level sub-problem. The search results corresponding to the first-level sub-problem, the first-level sub-problem and the second-level sub-problem are concatenated. The concatenated content is then searched again in the target database until no next-level sub-problem exists. The current search result is then used as the candidate answer.

8. A model question-answering optimization device, characterized in that, include: The determination module is used to determine the data extraction method for each knowledge data document based on the document type of each knowledge data document; The processing module is used to perform data extraction operations on each of the knowledge data documents according to the data extraction method, and store the extracted data into the corresponding database according to the document type. The question classification module is used to classify the question intent when a target question is received, and to obtain the question type of the target question; A splitting module is used to split the target problem into at least one sub-problem based on the problem type; A retrieval module is used to determine the target database corresponding to each sub-question and retrieve the candidate answers corresponding to the sub-question from the target database. The generation module is used to combine at least one of the sub-questions and their corresponding candidate answers, and then input them into a large model so that the large model generates the target answer corresponding to the target question.

9. A computer device, characterized in that, include: A processor and a memory, the processor being configured to execute a model question-answering optimization program stored in the memory to implement the model question-answering optimization method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the model question answering optimization method according to any one of claims 1 to 7.

Images