Power grid field table data reasoning and question answering method based on code enhancement framework
Through a method based on the code enhancement framework, a large language model is used to perform structural self-explanation and code generation on the WBS table data of power projects, which solves the problems of low efficiency and insufficient accuracy in the existing technology and realizes efficient and accurate WBS table data analysis and human resource estimation.
Patent Information
- Application Number
- CN202510854041.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-24
- Publication Date
- 2025-09-19
AI Technical Summary
Existing technologies are inefficient in power project WBS table data analysis and human resource estimation, are easily affected by human factors, have difficulty understanding complex structures and performing multi-step reasoning, and lack domain knowledge, resulting in inaccurate analysis results.
A method based on a code enhancement framework is adopted to perform structural self-explanation on the WBS table data of power projects through a large language model, generate executable code, and combine it with an external interpreter to execute and generate natural language answers, thereby improving the model's understanding of complex WBS table data and its ability to estimate headcount.
It achieves efficient WBS table data reasoning and headcount estimation under zero-sample conditions, improves the generalization ability and accuracy of large language models, and is suitable for work task decomposition and resource allocation in the field of power project management.
Smart Images

Figure CN120671849A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for reasoning and question-answering tabular data in the power grid field based on a code enhancement framework. The present invention relates to an application technology of a large language model based on a code enhancement framework in WBS tabular data analysis and human resource estimation. Background Art
[0002] Against the backdrop of the rapid informatization and digital transformation of the power industry, the power project management sector has accumulated a vast amount of WBS data. This data contains a wealth of critical information, including project task breakdown, man-hour consumption, and human resource requirements. Effectively leveraging this WBS data for intelligent analysis is crucial for improving project management efficiency, reducing project costs, and rationally allocating project resources. However, traditional WBS data analysis methods, such as manual analysis, rule-based systems, and simple database queries, are increasingly limited in the face of growing data volumes, increasingly complex WBS structures, and the emerging demand for intelligent resource estimation. Manual analysis is inefficient, costly, and susceptible to human error; rule-based systems lack flexibility and generalization, making them difficult to adapt to diverse project management analysis scenarios. Traditional database query languages, while capable of structured data retrieval, lack complex semantic understanding and deep reasoning, particularly when it comes to understanding user-defined work content and performing human resource estimation. In recent years, large language models have made significant progress in natural language understanding, knowledge reasoning, and code generation, opening up new opportunities for intelligent analysis of WBS data. However, directly applying general large language models to WBS table data analysis and human resource estimation still faces many challenges. First, WBS table data is often complex, containing multi-level headers and complex cell associations. General models have difficulty effectively understanding its inherent hierarchical structure and task decomposition semantic information. Second, WBS table data analysis and human resource estimation tasks often require multi-step reasoning and complex calculations, such as cross-row and cross-column data association and aggregation, as well as logical reasoning that incorporates project management domain knowledge. The reasoning ability and computational accuracy of general models need to be improved, especially for headcount estimation and resource optimization. In addition, general large language models may lack domain knowledge or have misunderstandings when processing specialized domain knowledge, which can affect the accuracy and reliability of analysis results, such as in understanding the terminology and specifications of industry-specific WBS tables. Therefore, how to effectively utilize large language models and overcome their shortcomings in processing complex table data and domain knowledge has become a key issue that needs to be addressed in the current field of intelligent WBS table data analysis and human resource estimation. This paper aims to propose a large language model power project WBS table data reasoning and question answering method based on a code enhancement framework to improve the performance and application value of the large language model in power project WBS table data analysis and human resource estimation tasks. Summary of the Invention
[0003] This paper proposes a method for reasoning and question-answering tabular data in the power grid field based on a code enhancement framework, which can effectively improve the understanding, reasoning and headcount estimation capabilities of large language models when processing complex WBS tabular data in the power grid operation and maintenance process.
[0004] The present invention adopts the following technical solutions.
[0005] A method for reasoning and question-answering tabular data in the power grid domain based on a code-augmented framework is proposed. This method aims to understand WBS tabular data and quickly estimate human resource requirements under zero-sample conditions. By constructing a code-augmented framework and utilizing a large language model to handle reasoning and question-answering tasks for WBS tabular data of power projects, the method first performs structural self-explanation on the WBS tabular data of power projects to improve the model's ability to understand the hierarchical structure and task decomposition relationships of complex WBS tables. Then, based on the table structure description and natural language questions formed by user-proposed work content descriptions, the method automatically generates executable code to extract relevant task information and resource configuration data from the WBS tabular data. The code is then executed through an external code interpreter to obtain results. Finally, the code execution results are combined with the model's own reasoning capabilities to generate natural language answers regarding the number of human resources required to complete specific work content.
[0006] The method comprises the following steps:
[0007] Step 1: Question Receive and Table Data Preparation. Specifically, the process involves receiving a natural language question about the power project WBS table data, which is a user-proposed work content description. The WBS table data containing multi-level headers is then obtained. The WBS table data must meet pre-set normalization requirements to ensure data consistency and parsability.
[0008] Step 2: The table structure is self-explanatory; specifically: First, in order to enable the large language model to effectively process the table data, the WBS table data that meets the standardization requirements is converted into a textual or structured representation that is easy for the large language model to understand and process, including serializing it into a CSV (comma separated value) string, Markdown table format, JSON object (where each object represents a row and the key is the column name), or a simplified HTML table code format. The format to be used can be flexibly selected according to the amount of data processing. Then, a large language model (such as DeepSeekV3) is used to perform structural analysis on this converted table data representation. The core of this step is to guide the model to generate a content-rich and structured natural language description of the table through prompt engineering;
[0009] Step 3: Code generation and information extraction. Specifically, based on the natural language question and the natural language description of the WBS table structure, the large language model generates code for information extraction. The code is used to extract rows and columns related to the work content proposed by the user from the WBS table data, and perform data filtering based on the WBS table structure and user requirements. Data filtering includes filtering tasks under specific work packages or aggregating and calculating the total working hours of a certain type of tasks.
[0010] Step 4: Code execution and result display; specifically, executing the generated code using an external code interpreter to obtain code execution results; displaying the code execution results in natural language to facilitate understanding and utilization by the large language model; the external code interpreter is a Python interpreter or an SQL execution engine;
[0011] Step 5: Answer inference and generation: Based on the natural language question, the natural language description of the WBS table structure, and the code execution results, the large language model infers the answer. Utilizing the large language model's reasoning capabilities, combined with numerical information such as working hours and number of people in the WBS table data, as well as semantic information such as task hierarchy and dependencies, the model infers and generates an estimate of the number of people required to complete the task. The estimate is presented as a natural language answer.
[0012] Step 6: Preprocessing the table data. Specifically, if the WBS table data volume is too large or the model input token length is limited, preprocess the table data in steps 1 and 2. The table data preprocessing method includes:
[0013] a) Identify the header rows and columns of the WBS table and extract header information such as "Work Package," "Task Name," "Work Hours," and "Number of Personnel." b) Filter out relevant WBS task items based on the header information and the work content specified by the user.
[0014] c) Integrate the filtered table data with the relevant cells in the original table data to maximize the retention of valid information and retain as much valid information as possible;
[0015] d) Control the token length of the pre-processed table data within the preset threshold range;
[0016] The table data preprocessing method includes: a cell integration method based on row expansion, column expansion or spiral expansion strategy.
[0017] In step 1, the preset normalization limits include:
[0018] Standardize table header information, including unified naming conventions and clear structure hierarchy;
[0019] Use clear methods to represent task hierarchical relationships, including using WBS codes or parent-child task IDs for identification;
[0020] Standardize the units of key numerical data and the format of text data; key numerical data include working hours and resource quantities; the tabular data includes information such as the project work breakdown structure, working hour estimates, and required number of people;
[0021] In step 1, the grid domain table data is the power project WBS table data, which includes the following types of data:
[0022] Basic project information, including project name, project number, project cycle, and project participants;
[0023] Work breakdown structure information, which records the hierarchical structure of the project's work packages, work tasks, and subtasks, as well as the task names, descriptions, and work content;
[0024] Man-hour estimation information, which records the planned man-hours, actual man-hours, and remaining man-hours for each work task or subtask, is used for schedule management and control;
[0025] Human resource information, which records the positions, skill requirements, and number of personnel required to complete each work task or subtask, for use in human resource allocation and cost estimation;
[0026] Task dependency information, which records the sequence, dependencies, and constraints between work tasks, is used for project schedule planning and optimization;
[0027] Project cost information, which records the cost budget, actual cost, deviation analysis, and other information for each work task or subtask, for project cost control and performance evaluation;
[0028] The natural language questions are replaced by natural language questions raised by users regarding the work content and headcount estimation in the WBS table of the power project, including: headcount estimation, such as "How many people are needed to complete the YY task in the XX work package?"; working hour query and estimation, such as "What are the planned working hours for the ZZ subtask?", "How many working hours are estimated to be required to complete the AA work content?"; resource allocation optimization, such as "How to adjust human resource allocation to shorten the project cycle?"; cost-benefit analysis, such as "Compare the labor costs and total costs of different construction plans?"; risk response measures query, such as "What are the response measures for the BB risk event?"
[0029] In step 2, the prompt word project specifically includes the following guidance strategies and contents:
[0030] a) Guide to identifying the overall information of the table: The prompt model first identifies and describes the overall subject or title of the table, for example: "This table is about the WBS breakdown structure and resource estimation of the XX project";
[0031] b) Multi-level header structure analysis guidance: For tables with multi-level headers, prompt the model to identify the header content level by level and describe its hierarchical relationship and coverage. For example, the prompt may include: "Please identify whether the table has multi-level headers. If so, please list the name of each level of headers and explain their parent-child or parallel relationship."
[0032] c) Key column semantic understanding guidance: Prompt the model to identify and explain the business meaning of key data columns, especially those related to task breakdown, work hours, number of people, dependencies, etc. For example: "Please find the columns representing 'Task ID', 'Task Name', 'Planned Work Hours', 'Required Number of People', and 'Predecessors', and explain the meaning and unit of each column (if applicable)."
[0033] d) Guidance on the correspondence between data cells and table headers: Prompts the model to understand how data rows correspond to the structure defined in the table header, ensuring that the model can associate specific data with its attributes.
[0034] e) Structured output format guidance: Requires the model to output the above information according to a predefined structure or template (such as JSON format or specific point-to-point column description). For example, "Please output the table header structure, key columns, and their semantics in JSON format."
[0035] The prompt words in the prompt word project guide the model to output natural language descriptions that include information about table header hierarchies, parent-child relationships, and key column semantics. These descriptions are structured and standardized, making them easier to understand and utilize in subsequent steps.
[0036] In order to further improve the efficiency and accuracy of subsequent processing, the natural language description output by the large language model is subjected to structured parsing. Specifically, keyword extraction, regular expression matching, or a rule-based parser is used to extract standardized table metadata from the natural language description of the model. The table metadata is structured table metadata, including: a list of table headers and their hierarchy, clear identification of each key column and its semantic label, and a representation of the task hierarchy relationship; it is used to directly serve the subsequent code generation and answer inference steps; to ensure the accuracy and deconstructibility of the description, the correctness of the parser's parsing results is confirmed through rule verification (such as checking whether the hierarchical relationship conforms to the WBS specification), comparison with known templates, comparison and verification of its output with predefined standard data structure patterns, evaluation of its support effect on the performance of downstream tasks (such as code generation), or manual sampling review, thereby ensuring the effectiveness of subsequent processing based on this description;
[0037] In step 2, the table structure self-explanation step further includes: the large language model first identifies the title of the WBS table, and understands that the table is about the theme and content summary of the work breakdown structure and resource estimation of the power project; then, identifies the table headers, including row headers and column headers, and understands the meaning and data dimensions represented by each header, such as "work package", "task name", "planned working hours", "required number of people", etc.; further, parses the hierarchical relationship of the multi-level headers of the WBS table, such as identifying the inclusion relationship and logical relationship between work packages, work tasks, and subtasks, and constructs a WBS task hierarchical tree structure; then, analyzes the correspondence between the headers and data cells, and determines the header category and dimension to which each data cell belongs, such as identifying whether the cell data is a task name, working hours value, or number of people value; finally, based on the above parsing results, generates a natural language description of the WBS table structure;
[0038] The natural language description includes the hierarchical header structure of the table, such as "This table describes the WBS of an electric power project, which includes three levels: work packages, work tasks, and subtasks", the meaning of each header, such as "The 'Planned Working Hours' column represents the planned working hours required to complete the task", and the semantic association between the header and the data cell, such as "The data in the 'Required Number of People' column is associated with the task entry in the 'Task Name' column, indicating the number of people required to complete the corresponding task", which is used to assist the large language model in understanding the organization and intrinsic meaning of the table data, improve the accuracy of subsequent information extraction and reasoning, and especially assist in understanding the relationship between the table structure and the number of people estimation task.
[0039] In step three, the code generation and information extraction steps include: determining the code type to be executed using a large language model based on the intent of the natural language question and the natural language description of the WBS table structure, wherein the code type includes but is not limited to Python scripts or SQL query statements; based on the determined code type, the large language model generates specific code instructions, wherein the code instructions include selection statements for locating relevant work packages, work tasks, or subtasks in the WBS table data, and functions or methods for performing data operations; the data operations are replaced with operations related to the WBS table data, including:
[0040] Task filtering: filtering WBS tasks based on specific conditions. Task filtering includes filtering all tasks under a specific work package or filtering tasks on the critical path.
[0041] Task sorting: sorting WBS tasks by one or more columns of data, such as sorting tasks by planned working hours or task priority;
[0042] Data aggregation: performing statistical calculations on WBS table data, such as calculating the total working hours of a work package or the average number of people required for a certain type of task;
[0043] Task dependency analysis analyzes the dependencies between WBS tasks, such as finding the predecessor and successor tasks of a task. The generated code is designed to accurately extract the information needed to answer natural language questions about work content and headcount estimation from WBS table data, and organize the extracted information into structured data to facilitate subsequent answer inference and generation, ultimately used for headcount estimation reasoning.
[0044] Step 4 includes code execution and result display, which includes: using an external Python interpreter or SQL execution engine to execute the code generated in step 3 to obtain the result after code execution, which is a structured WBS table data subset or value, such as a filtered task list, aggregated total working hours or average number of people; converting the code execution result into a natural language description, which is a summary and explanation of the code execution result, for example, converting the task list under a specific filtered work package into "The following are the tasks included in the XX work package: Task A, Task B, Task C..."; displaying the code execution result and the natural language description to the large language model at the same time so that the large language model can understand the specific process and results of the code execution and make subsequent answer inferences based on the code execution results; the purpose of displaying the code execution results is to: verify the correctness of the code and ensure the accuracy of information extraction, especially for the information extraction task of WBS table data; assist the large language model in understanding the WBS table data and task breakdown structure, reducing the risk of hallucinations; and provide reliable data support for the generation of the final headcount estimation answer.
[0045] In step 5, the answer inference and generation step further includes: the large language model receives the natural language question, the natural language description of the WBS table structure, and the code execution results, and combines its pre-trained knowledge and reasoning ability to deeply understand and analyze the question, focusing on understanding the work content description proposed by the user and the relationship between tasks and number of people in the WBS table;
[0046] Based on its understanding of the problem and the data support provided by the code execution results, the large language model performs multi-step reasoning. This multi-step reasoning includes: analyzing the correlation between the code execution results and the natural language question to determine whether the code execution results can directly answer the headcount estimation question; if the code execution results cannot directly answer the headcount estimation question, further analyzing the deep semantics and potential information contained in the code execution results, such as analyzing the relationship between different task types, task complexity, and the required number of people; combining the natural language description of the WBS table structure to understand the impact of factors such as task hierarchy and dependencies on headcount estimation; and gradually deriving the final answer using a reasoning chain. The reasoning chain includes intermediate reasoning steps and logical deduction processes, such as "First, determine the task complexity based on the task type; then, refer to the headcount configuration of similar tasks in the WBS table; then, consider the dependencies and parallelism between tasks; finally, based on all the above factors, estimate the number of people required to complete the task." Finally, the answer obtained by reasoning is organized and expressed in natural language form to generate a final natural language answer. This natural language answer concisely, clearly, and accurately answers the work content proposed by the user and provides a reasonable headcount estimate result.
[0047] In step 6, the table data preprocessing step is specifically as follows: when the original power project WBS table data contains a large number of rows or columns, or the token length of the table data exceeds the context window limit of the large language model, in order to ensure that the model can effectively process the table data and reduce computing resource consumption, the table data preprocessing is performed;
[0048] The table data preprocessing adopts an adaptive cell integration algorithm to compress the table data token length while retaining the key information in the table data as much as possible, especially the task information and resource information related to the headcount estimation;
[0049] The cell integration algorithm includes:
[0050] The strategy based on row expansion prioritizes the integration of cells in the same row, assuming that WBS task items in the same row have higher semantic relevance. The strategy based on column expansion prioritizes the integration of cells in the same column, assuming that cells in the same column describe the same type of attribute information, such as working hours or number of people.
[0051] The spiral expansion strategy is to expand and integrate cells in a spiral manner, taking into account information in both row and column directions to more comprehensively preserve the structure and content of the WBS table;
[0052] In step six, the cell integration algorithm dynamically adjusts the integration strategy and strength based on the preset token length threshold to ensure that the token length of the preprocessed table data meets the model input requirements while preserving the integrity and valid information of the table data to the greatest extent possible, providing support for the subsequent WBS table structure self-explanation, code generation, and headcount estimation answer inference steps.
[0053] In step 6, when the original WBS table data, after being converted into textual or structured representation according to the method in step 2, has a token length exceeding the context window limit of the large language model (e.g., common 4096, 8192, 16K or higher tokens, depending on the selected large language model), or in order to improve data processing efficiency and reduce computing resource consumption, the table data preprocessing step is used to intelligently compress and streamline the table data;
[0054] The core goal of tabular data preprocessing is to ensure that, while strictly adhering to the model's token budget, a series of strategic operations are performed to ensure that the data input to the large language model retains as much information as possible that is critical for answering user questions and completing subsequent reasoning tasks (such as headcount estimation), while effectively avoiding the loss of key contextual information caused by simple data truncation.
[0055] Methods and strategies for preprocessing tabular data include:
[0056] (1) Intelligent Filtering and Pruning, specifically:
[0057] a. Dynamic filtering based on question relevance: Analyze the semantics of the user's natural language questions to identify core entities, intent, and scope. Based on this, dynamically filter out rows (task items), columns (attribute fields), or specific data subsets in the WBS table that are directly related to the question topic. For example, if a user asks about the number of people in a work package at a specific project phase, the system will prioritize extracting relevant data columns for this work package and its direct subtasks within that project phase (such as task ID, task name, planned hours, required number of people, predecessor tasks, etc.).
[0058] b. Removal of structurally irrelevant information: Systematically remove columns in the table that contribute little or are irrelevant to the core WBS analysis (task decomposition, time estimation, resource allocation, dependency identification, etc.), such as format control characters, non-business-related comment columns, columns with a high proportion of null values, or auxiliary information columns that do not meet the current analysis objectives.
[0059] (2) Data Aggregation and Summarization, specifically:
[0060] a. Hierarchical aggregation of numerical data: For WBS tables with a clear hierarchical structure (e.g., work package-work task-subtask), numerical data (e.g., planned hours, actual hours, required headcount) of sub-tasks can be meaningfully aggregated (e.g., sum, weighted average, maximum / minimum values, etc.) to the parent task. This significantly reduces the number of data entries while preserving macro trends and total information. For example, summing the "required headcount" of all subtasks can yield the "total required headcount" of their direct parent task.
[0061] b. Descriptive text summary: For cells containing long descriptive text (such as detailed task descriptions, notes, etc.), a lightweight text summarization algorithm can be used or specially designed prompts can be used to guide the large language model itself (possibly using a model with a smaller context window or block summary) to generate a short summary of its core content, thereby significantly reducing its token usage.
[0062] (3) Chunking and Iterative Information Integration, specifically:
[0063] a. Logical chunking or token budget chunking: Split large table data into multiple independent but potentially logically related data chunks or sub-tables based on their inherent logical structure (e.g., by major work packages, by project phases, by major dependency chains) or a preset token limit (e.g., a value slightly less than the model's maximum token limit);
[0064] b. Block processing and result fusion: Each data block is fed into the large language model for analysis, extracting key information, forming preliminary judgments, or calculating intermediate results. Subsequently, a result integration and comprehensive reasoning mechanism is designed (this mechanism itself can also leverage the reasoning capabilities of the large language model) to aggregate, compare, disambiguate, correlate, and comprehensively reason the analysis results from different data blocks to form a comprehensive understanding of the entire original table and the final answer. For example, when estimating the total number of people, the estimated number of people in each block is summed up or more complex dependency adjustments are performed.
[0065] (4) Sliding Window Approach, specifically:
[0066] When the table data is completely serialized into a long text (such as converted to Markdown or JSON string), the sliding window technique can be used. A window of fixed size (in tokens) is set, which slides gradually over the serialized text. The model processes the text fragment within the current window at each window position. To maintain contextual coherence, a certain overlapping area can be set between adjacent windows. The model processes the content of each window and gradually accumulates or transmits the processing results (such as extracted information and intermediate conclusions) to build an understanding of the entire table content.
[0067] (5) Optimized cell integration and information density maintenance strategies, specifically:
[0068] Apply the cell integration algorithm based on row expansion, column expansion or spiral expansion in step 6; during the integration process, introduce an information value or information density assessment mechanism; for example, prioritize retaining cell contents containing key numerical data, specific business keywords, or those judged to be more relevant based on the user's question, while cell contents with high repetition or low information content may be appropriately compressed or omitted during the integration. (6) Embedding-based contextual retrieval augmentation, specifically: semantically embed the rows of the table, meaningful cell combinations or pre-defined sub-tables (data blocks), and store these embedding vectors and their corresponding original data fragments in the vector database. When the user asks a question, the user's question is first converted into an embedding vector, and then the most relevant table data fragments are found in the vector database through similarity search. These retrieved, highly relevant fragments are then combined (within the token limit) as contextual input for the large language model to perform final reasoning and answer.
[0069] The above preprocessing method enhances the processing capability, accuracy, and robustness of WBS tables for power projects containing massive amounts of data and extremely complex structures, ensuring that key information can be effectively extracted and utilized even under strict token length restrictions, providing users with valuable analysis results and problem answers.
[0070] The method further includes step seven, performing a model performance evaluation, namely, evaluating the performance of the method, including the accuracy of the answers and their relevance to natural language questions, as well as the efficiency of code generation and execution; the evaluation is performed using a standard test dataset containing power project WBS table data, user task descriptions, and headcount estimation results;
[0071] The model performance evaluation in step seven is specifically as follows: In order to comprehensively and objectively evaluate the effectiveness of the method in the WBS table data reasoning and question-answering tasks of power projects, an in-depth evaluation is conducted from multiple dimensions using a typical application scenario (headcount estimation);
[0072] First, regarding answer accuracy, for key tasks like headcount estimation, the accuracy of headcount estimation results will be used as a core metric, combined with manual evaluation to comprehensively measure the quality of the answers generated by the model. The headcount estimation accuracy focuses on assessing the degree of consistency between the model's predicted headcount for this type of task and the actual headcount or expert estimates in the WBS. Meanwhile, manual evaluation focuses on determining the rationality and feasibility of various answers, including headcount estimation, from the perspective of project management and resource allocation, thus compensating for the shortcomings of automated evaluation metrics in terms of domain knowledge and practical experience.
[0073] Secondly, in terms of answer relevance, we will evaluate whether the natural language answers generated by the model can effectively answer users' questions about work content and headcount estimation, whether the answers are relevant, whether they include key information of concern to users such as task breakdown structure, time estimates, resource requirements, and whether the answers are expressed clearly, fluently, and easily understood. Relevance is evaluated manually, with project management experts or senior engineers scoring the answers generated by the model and evaluating them based on multiple dimensions, such as the completeness of the answers, the amount of information, practicality, and the naturalness of the language.
[0074] In terms of code generation and execution efficiency, the focus will be on evaluating the quality of the model-generated code, including its executable nature, logical correctness, and efficiency, particularly its ability to accurately extract task information from the WBS table and perform calculations related to headcount estimation. The time required for code execution and the response speed of the entire question-and-answer system will also be evaluated to measure the efficiency and real-time performance of the proposed method in practical applications. Efficiency evaluation will be conducted using automated testing and performance analysis tools, coupled with stress testing in conjunction with actual application scenarios, to fully understand the performance bottlenecks and optimization potential of the adopted model.
[0075] In terms of text classification accuracy evaluation, the confusion matrix is used to record the performance of the model, including precision P, recall R and F1 value as evaluation indicators, which are calculated through the confusion matrix.
[0076] The confusion matrix of the classification results is expressed in a table as follows:
[0077]
[0078] P refers to the proportion of samples predicted as positive by the classifier and correctly predicted to all samples predicted as positive. The calculation formula is as follows:
[0079]
[0080] R refers to the proportion of samples predicted as positive by the classifier and predicted correctly to all truly positive samples. The calculation formula is as follows:
[0081]
[0082] F1 is a comprehensive indicator of P and R, and its calculation formula is as follows:
[0083]
[0084] In terms of answer accuracy evaluation, for the headcount estimation task, the headcount estimation accuracy and exact match rate (EM) will be used as evaluation indicators. The headcount estimation accuracy focuses on evaluating the degree of consistency between the number of people predicted by the model and the actual number of people or the expert estimate in the WBS table. The root mean square error (RMSE) or mean absolute percentage error (MAPE) indicators are used to quantify the deviation of the headcount estimation. Lower RMSE and MAPE values indicate higher headcount estimation accuracy.
[0085] For non-numeric answers, such as task names and resource types, the exact match rate evaluation model is used to return the proportion of answers that are completely consistent with the standard answer. The higher the EM value, the more accurate the answer.
[0086] To more closely analyze the types of model errors, we roughly categorized the questions by question type (e.g., headcount estimation, task information query, etc.). For each type of question, we analyzed the model's prediction results, counting the true positives (TP) and false negatives (FN). We also calculated precision, recall, and F1-score to help analyze the model's performance on different types of questions.
[0087] To ensure the objectivity and comparability of the evaluation results, a standard test dataset containing power project WBS table data, various types of natural language questions (including but not limited to task descriptions and headcount estimation requests), and corresponding standard answers (such as headcount estimation results) will be selected as the test benchmark. Furthermore, an evaluation dataset containing natural language questions of varying types and difficulty levels regarding work content and headcount estimation will be constructed to comprehensively examine the adaptability and robustness of the model in different scenarios.
[0088] When evaluating the results, the method will be compared with existing advanced table data analysis and project management tools, such as traditional spreadsheet software, project management software, and other artificial intelligence-based resource estimation models, to fully demonstrate the advantages and innovations of the method in WBS table data reasoning and headcount estimation.
[0089] The large language model is DeepSeek V3.
[0090] The present invention aims to overcome the shortcomings of the existing technology in the analysis of WBS table data and human resource estimation of power projects. It proposes a method for reasoning and question-answering of WBS table data of power projects using a large language model based on a code enhancement framework. The method can effectively improve the understanding, reasoning and headcount estimation capabilities of the large language model when processing complex WBS table data.
[0091] The present invention constructs a multi-step code enhancement framework to guide the large language model to gradually complete tasks such as understanding WBS table data, extracting task information, executing code, and generating answers for headcount estimation, thereby realizing intelligent analysis of WBS table data for power projects and rapid estimation of human resource needs. The method of the present invention does not require manually labeled example data for model training, and can achieve efficient WBS table data reasoning and headcount estimation under zero-sample conditions. By strictly normalizing and pre-processing the input WBS table data, and designing targeted prompt word engineering to interact with the large language model, while performing structured analysis and utilization of the model's output, the present invention can effectively process complex WBS table data containing multi-level headers, has good generalization ability and practical value, and is particularly suitable for work task decomposition and resource allocation scenarios in the field of power project management.
[0092] The present invention proposes a large language model power project WBS table data reasoning and question-answering method based on a code enhancement framework, which can improve the performance and application value of the large language model in power project WBS table data analysis and human resource estimation tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0093] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0094] Attachment Figure 1 This is a schematic diagram of the table structure analysis prompt words of the present invention;
[0095] Attachment Figure 2 It is a schematic diagram of the architecture algorithm of the method of the present invention;
[0096] Attachment Figure 3 Schematic diagram of model evaluation of the method of the present invention. DETAILED DESCRIPTION
[0097] As shown in the figure, a method for reasoning and question-answering tabular data in the power grid field based on a code enhancement framework is shown. The method aims to understand WBS tabular data and quickly estimate human resource requirements under zero-sample conditions. By constructing a code enhancement framework and utilizing a large language model to process reasoning and question-answering tasks for WBS tabular data of power projects, the method first performs structural self-explanation on the WBS tabular data of power projects to improve the model's ability to understand the hierarchical structure and task decomposition relationships of complex WBS tables. Then, based on the table structure description and natural language questions formed by the user's work content description, the method automatically generates executable code, extracts relevant task information and resource configuration data from the WBS tabular data, and executes the code through an external code interpreter to obtain results. Finally, the method combines the code execution results with the model's own reasoning ability to generate a natural language answer about the number of human resources required to complete specific work content.
[0098] The method comprises the following steps:
[0099] Step 1: Question reception and form data preparation;
[0100] In this example, a large language model is used to perform intelligent reasoning and question answering on the WBS table data of a power project. The process begins with the user submitting their natural language question and the relevant WBS table data through an interactive interface.
[0101] (1) Design of user interface and data input method:
[0102] a. The present invention can be integrated into a standalone desktop application, a web application, or embedded as a plug-in into existing project management software (such as Microsoft Project, Primavera P6, etc.) or enterprise information systems.
[0103] b. Users enter their questions in natural language through this interface, such as "How many people are needed to complete the civil engineering portion of XX substation?" or "Estimate the labor cost of YY work package for the line project."
[0104] c. At the same time, users upload WBS spreadsheet data files through this interface. The system design should prioritize and encourage users to submit machine-readable, structured spreadsheet file formats. These formats include but are not limited to:
[0105] a) CSV (Comma-Separated Values): Plain text format, easy to parse.
[0106] b) TSV (Tab-Separated Values): Similar to CSV, separated by tabs.
[0107] c) XLSX (Microsoft Excel Open XML Format): A modern Excel file format based on XML and highly structured.
[0108] d) XLS (Microsoft Excel Binary File Format): an older version of Excel file format.
[0109] e) ODS (OpenDocument Spreadsheet): a spreadsheet in open document format.
[0110] d. For formats that are not directly machine-readable (such as scanned PDFs, tables in picture format): In principle, the method of the present invention does not directly process such formats, because they lack structured information, and directly inputting them into the large language model is not effective. If the user submits such a format, the system will prompt the user to convert it to the recommended structured format mentioned above, or, if the system has integrated OCR (optical character recognition) function, it will attempt to perform OCR conversion. However, it should be emphasized that the accuracy of the OCR conversion will directly affect the subsequent processing effect, and the data after OCR conversion still needs further structured processing to adapt to the process of the present invention. Therefore, directly submitting a structured spreadsheet is the first choice.
[0111] (2) Reading, preliminary analysis and adaptation of table data to LLM:
[0112] a. Once the user submits a spreadsheet file, the system backend calls the corresponding file parsing library (for example, the pandas library in Python for processing CSV, XLSX, etc., or a dedicated API interface) to read and parse the spreadsheet data content.
[0113] b. After the data is read into the program memory, the system will perform a series of interface adaptation and normalization preprocessing, including:
[0114] a) Standardization of header information (consistent naming and clear structure).
[0115] b) Clear and consistent representation of hierarchical relationships (e.g. WBS code, parent task ID).
[0116] c) Strict unification of data content and format (clear units, standardized date formats, and normalized enumeration values).
[0117] d) Guarantee of file format and data integrity.
[0118] c. Key Step: Convert to a Large Language Model-Friendly Input Format: After completing the above normalization and before entering "Step 2: Table Structure Self-Explanation," the system must convert the structured table data in memory into one or more textual or serialized formats that can be effectively processed by large language models. This includes:
[0119] a) Serialize to CSV string: Convert the entire table or selected parts of it into comma-delimited text lines.
[0120] b) Markdown table format: a human-readable and clearly structured text markup format.
[0121] c) JSON object array: Each row of data is represented as a JSON object, with the key being the column name.
[0122] d) Simplified HTML table code: only keep the core , , , etc. tags.
[0123] e) Other custom linearized text representations.
[0124] This transformation step is the bridge between the raw tabular data and the understanding capabilities of large language models.
[0125] The tabular data includes information such as the project work breakdown structure, work hour estimates, and required number of people;
[0126] Step 1 specifically includes: receiving a natural language question regarding the WBS table data of the power project, wherein the natural language question is a work content description proposed by the user; then obtaining the WBS table data of the power project including multi-level headers, wherein the WBS table data must meet preset normalization restrictions to ensure data consistency and parsability;
[0127] Step 2: The table structure is self-explanatory; specifically: First, in order to enable the large language model to effectively process the table data, the WBS table data that meets the standardization requirements is converted into a textual or structured representation that is easy for the large language model to understand and process, including serializing it into a CSV (comma separated value) string, Markdown table format, JSON object (where each object represents a row and the key is the column name), or a simplified HTML table code format. The format to be used can be flexibly selected according to the amount of data processing. Then, a large language model (such as DeepSeekV3) is used to perform structural analysis on this converted table data representation. The core of this step is to guide the model to generate a content-rich and structured natural language description of the table through prompt engineering;
[0128] Step 3: Code generation and information extraction. Specifically, based on the natural language question and the natural language description of the WBS table structure, the large language model generates code for information extraction. The code is used to extract rows and columns related to the work content proposed by the user from the WBS table data, and perform data filtering based on the WBS table structure and user requirements. Data filtering includes filtering tasks under specific work packages or aggregating and calculating the total working hours of a certain type of tasks.
[0129] Step 4: Code execution and result display; specifically, executing the generated code using an external code interpreter to obtain code execution results; displaying the code execution results in natural language to facilitate understanding and utilization by the large language model; the external code interpreter is a Python interpreter or an SQL execution engine;
[0130] Step 5: Answer inference and generation: Based on the natural language question, the natural language description of the WBS table structure, and the code execution results, the large language model infers the answer. Utilizing the large language model's reasoning capabilities, combined with numerical information such as working hours and number of people in the WBS table data, as well as semantic information such as task hierarchy and dependencies, the model infers and generates an estimate of the number of people required to complete the task. The estimate is presented as a natural language answer.
[0131] Step 6: Preprocessing the table data. Specifically, if the WBS table data volume is too large or the model input token length is limited, preprocess the table data in steps 1 and 2. The table data preprocessing method includes:
[0132] a) Identify the header rows and columns of the WBS table and extract header information such as "Work Package," "Task Name," "Management Hours," and "Number of Personnel." b) Filter out relevant WBS task items based on the header information and the work content specified by the user.
[0133] c) Integrate the filtered table data with the relevant cells in the original table data to maximize the retention of valid information and retain as much valid information as possible;
[0134] d) Control the token length of the pre-processed table data within the preset threshold range;
[0135] The table data preprocessing method includes: a cell integration method based on row expansion, column expansion or spiral expansion strategy.
[0136] In step 1, the preset normalization limits include:
[0137] Standardize table header information, including unified naming conventions and clear structure hierarchy;
[0138] Use clear methods to represent task hierarchical relationships, including using WBS codes or parent-child task IDs for identification;
[0139] Standardize the units of key numerical data and the format of text data; key numerical data include working hours and resource quantities; the tabular data includes information such as the project work breakdown structure, working hour estimates, and required number of people;
[0140] In step 1, the grid domain table data is the power project WBS table data, which includes the following types of data:
[0141] Basic project information, including project name, project number, project cycle, and project participants;
[0142] Work breakdown structure information, which records the hierarchical structure of the project's work packages, work tasks, and subtasks, as well as the task names, descriptions, and work content;
[0143] Man-hour estimation information, which records the planned man-hours, actual man-hours, and remaining man-hours for each work task or subtask, is used for schedule management and control;
[0144] Human resource information, which records the positions, skill requirements, and number of personnel required to complete each work task or subtask, for use in human resource allocation and cost estimation;
[0145] Task dependency information, which records the sequence, dependencies, and constraints between work tasks, is used for project schedule planning and optimization;
[0146] Project cost information, which records the cost budget, actual cost, deviation analysis, and other information for each work task or subtask, for project cost control and performance evaluation;
[0147] The natural language questions are replaced with natural language questions raised by users regarding the work content and headcount estimation in the WBS table of the power project, including: headcount estimation, such as "How many people are needed to complete the YY task in the XX work package?"; working time query and estimation, such as "What are the planned working hours for the ZZ subtask?", "How many working hours are estimated to be required to complete the AA work content?"; resource allocation optimization, such as "How to adjust human resource allocation to shorten the project cycle?"; cost-benefit analysis, such as "Compare the labor costs and total costs of different construction plans?"; risk response query, such as "What are the response measures for the BB risk event?"
[0148] In this example, in step 2, the method innovatively introduces a self-explanatory step to address the complex hierarchical structure commonly found in power project WBS data (e.g., nested work packages, tasks, and subtasks). This step leverages the natural language understanding capabilities of a large language model and, through carefully designed prompt engineering, guides the model to conduct in-depth analysis of the input WBS data that meets standardized requirements.
[0149] In step 2, the prompt word project specifically includes the following guidance strategies and contents:
[0150] a) Guide to identifying the overall information in the table: Prompt the model to first identify and describe the overall subject or title of the table, for example: "This table is about the WBS breakdown structure and resource estimation of the XX project."
[0151] b) Multi-level header structure analysis guidance: For tables containing multi-level headers, prompt the model to identify the header content level by level and describe its hierarchical relationship and coverage. For example, the prompt word may include: "Please identify whether the table has multi-level headers. If so, please list the name of each level of header and explain their parent-child or parallel relationship."
[0152] c) Key column semantic understanding guidance: Prompt the model to identify and explain the business meaning of key data columns, especially those related to task breakdown, work hours, number of people, dependencies, etc. For example: "Please find the columns representing 'Task ID', 'Task Name', 'Planned Work Hours', 'Required Number of People', and 'Predecessors', and explain the meaning and unit of each column (if applicable)."
[0153] d) Guidance on the correspondence between data cells and table headers: Prompts the model to understand how data rows correspond to the structure defined in the table header, ensuring that the model can associate specific data with its attributes.
[0154] e) Structured output format guidance: Requires the model to output the above information according to a predefined structure or template (such as JSON format or specific point-to-point column description). For example: "Please output the table header structure, key columns, and their semantics in JSON format."
[0155] The prompt words in the prompt word project guide the model to output natural language descriptions that include information about table header hierarchies, parent-child relationships, and key column semantics. These descriptions are structured and standardized, making them easier to understand and utilize in subsequent steps.
[0156] In order to further improve the efficiency and accuracy of subsequent processing, the natural language description output by the large language model is subjected to structured parsing. Specifically, keyword extraction, regular expression matching, or a rule-based parser is used to extract standardized table metadata from the natural language description of the model. The table metadata is structured table metadata, including: a list of table headers and their hierarchy, clear identification of each key column and its semantic label, and a representation of the task hierarchy relationship; it is used to directly serve the subsequent code generation and answer inference steps; to ensure the accuracy and deconstructibility of the description, the correctness of the parser's parsing results is confirmed through rule verification (such as checking whether the hierarchical relationship conforms to the WBS specification), comparison with known templates, comparison and verification of its output with predefined standard data structure patterns, evaluation of its support effect on the performance of downstream tasks (such as code generation), or manual sampling review, thereby ensuring the effectiveness of subsequent processing based on this description;
[0157] In this example, this type of prompt engineering guides the model to automatically identify multi-level headers in a table (for example, in response to prompt b, it identifies "work package," "task name," "planned work hours," "required number of people," etc.) and understand the hierarchical relationships and semantic associations between headers (for example, in response to prompts b and c, the model understands that "work package" contains "work tasks," "subtasks" belong to "work tasks," and that "'planned work hours' and 'required number of people' are resource descriptions for 'work tasks'"). The natural language description or structured data it outputs is more structured and standardized, making it easier to understand and use in subsequent steps.
[0158] Specifically, under the guidance of the above prompt word project, the large language model will:
[0159] 1. Respond to the "Table overall information identification guidance" (such as point a), analyze the table content, and generate a preliminary description such as "This table is about the WBS breakdown structure and resource estimation of the XX project."
[0160] 2. In response to the "Multi-level header structure parsing guidance" (as in point b), the model will identify, based on its understanding of the input form (not limited to HTML, but can be any structured or semi-structured representation), that the first row is "Work Package," the second row is "Work Task," and the accompanying "Planned Man Hours," "Required Headcount," and so on. If there is deeper nesting, such as "Subtasks" under "Work Task," the model will also identify and describe this hierarchical tree structure, for example, forming a hierarchical understanding of "Work Package - Work Task - Subtask."
[0161] 3. In response to the "Key Column Semantic Understanding Guidance" (such as point c), the model will locate, for example, the column named "Task Name" and understand based on the prompt that it represents the specific description of the WBS item; locate the column named "Planned Man-Hours" and, combined with the unit information required in the prompt (if provided), understand that it is the estimated man-hours required to complete the task; similarly, understand that the "Required Number of People" column represents the manpower required to complete the task.
[0162] 4. Finally, according to the "Structured Output Format Guidance" (e.g., point e), the model will output the table structure information parsed in response to prompts a through d in the specified natural language description format or structured data (e.g., JSON). For example, if the prompt requires a natural language description, the model might output: "This WBS table contains a three-level hierarchy: work packages, tasks, and subtasks. The 'Task Name' column describes the specific work content, the 'Planned Man-Hours' column indicates the planned man-hours to complete the task, and the 'Required Personnel' column indicates the number of people required to complete the task."
[0163] This natural language description or structured data can effectively enhance the large language model's understanding of the task decomposition structure and resource allocation semantics inherent in the WBS table data. To ensure the accuracy and deconstructibility of the description, the correctness of its parsing results can be confirmed through rule verification (such as checking whether the hierarchical relationship complies with the WBS specification), comparison with known templates, comparison and verification of its output with predefined standard data structure patterns, evaluation of its support effect on the performance of downstream tasks (such as code generation), or manual sampling review, thereby ensuring the effectiveness of subsequent processing based on this description. In this way, important structured knowledge is provided for the subsequent code generation and answer inference steps, especially for understanding the correspondence between the work content proposed by the user and the task items in the WBS table, as well as the correlation between the table structure and the headcount estimation task, providing key support.
[0164] In addition, to further improve the efficiency and accuracy of subsequent processing, this step can also include structured parsing of the description output by the large language model (especially in natural language). For example, keyword extraction, regular expression matching, or a rule-based parser can be used to extract standardized table metadata from the model's natural language description, such as: a list of table headers and their hierarchy, clear identification of each key column and its semantic label, and a representation of task hierarchical relationships. This structured metadata will directly serve the subsequent code generation and answer inference steps.
[0165] In step 2, the table structure self-explanation step further includes: the large language model first identifies the title of the WBS table, understanding that the table is the theme and content summary of the work breakdown structure and resource estimation of the power project; then, it identifies the table headers, including row headers and column headers, and understands the meaning and data dimensions represented by each header, such as "work package", "task name", "planned working hours", "required number of people", etc.; further, it parses the hierarchical relationship of the multi-level headers of the WBS table, such as identifying the inclusion relationship and logical relationship between work packages, work tasks, and subtasks, and constructing a WBS task hierarchical tree structure; then, it analyzes the correspondence between the table headers and data cells, and determines the table header category and dimension to which each data cell belongs, such as identifying whether the cell data is a task name, working hours value, or number of people value; finally, based on the above parsing results, it generates a natural language description of the WBS table structure;
[0166] The natural language description includes the hierarchical header structure of the table, such as "This table describes the WBS of an electric power project, which includes three levels: work packages, work tasks, and subtasks", the meaning of each header, such as "The 'Planned Working Hours' column represents the planned number of working hours required to complete the task", and the semantic association between the header and the data cell, such as "The data in the 'Required Number of People' column is associated with the task entry in the 'Task Name' column, indicating the number of people required to complete the corresponding task", which is used to assist the large language model in understanding the organization and intrinsic meaning of the table data, improve the accuracy of subsequent information extraction and reasoning, and especially assist in understanding the relationship between the table structure and the number of people estimation task.
[0167] In this example, the prompt word engineering in step 2 specifically includes guiding the model to identify and describe the table title, multi-level header structure, hierarchical relationships between headers (such as parent-child relationships and parallel relationships), semantic information of key data columns (such as task name, working hours, and number of people), and the correspondence between data cells and headers. The model can also be required to output description content containing the above information according to a predefined structure or template, so that the natural language description output by the model is more structured and standardized, which is convenient for understanding and utilization in subsequent steps. The step also includes structural parsing of the natural language description output by the model to extract key table structure information;
[0168] To ensure the accuracy and deconstructibility of the description, the correctness of its parsing results can be confirmed through rule verification (such as checking whether the hierarchical relationship conforms to the WBS specification), comparison with known templates, comparison and verification of its output with predefined standard data structure patterns, evaluation of its support effect on the performance of downstream tasks (such as code generation), or manual sampling review, so as to ensure the effectiveness of subsequent processing based on this description;
[0169] In step three, the code generation and information extraction steps include: determining the code type to be executed using a large language model based on the intent of the natural language question and the natural language description of the WBS table structure, wherein the code type includes but is not limited to Python scripts or SQL query statements; based on the determined code type, the large language model generates specific code instructions, wherein the code instructions include selection statements for locating relevant work packages, work tasks, or subtasks in the WBS table data, and functions or methods for performing data operations; the data operations are replaced with operations related to the WBS table data, including:
[0170] Task filtering: filtering WBS tasks based on specific conditions. Task filtering includes filtering all tasks under a specific work package or filtering tasks on the critical path.
[0171] Task sorting: sorting WBS tasks by one or more columns of data, such as sorting tasks by planned working hours or task priority;
[0172] Data aggregation: performing statistical calculations on WBS table data, such as calculating the total working hours of a work package or the average number of people required for a certain type of task;
[0173] Task dependency analysis analyzes the dependencies between WBS tasks, such as finding the predecessor and successor tasks of a task. The generated code is designed to accurately extract the information needed to answer natural language questions about work content and headcount estimation from WBS table data, and organize the extracted information into structured data to facilitate subsequent answer inference and generation, ultimately used for headcount estimation reasoning.
[0174] Step 4 includes code execution and result display, which includes: using an external Python interpreter or SQL execution engine to execute the code generated in step 3 to obtain the results after code execution, which are a structured WBS table data subset or value, such as a filtered task list, aggregated total working hours, or average number of people; converting the code execution results into a natural language description, which summarizes and explains the code execution results, for example, converting the task list under a specific filtered work package into "The following are the tasks included in the XX work package: Task A, Task B, Task C..."; displaying the code execution results and natural language description to the large language model at the same time so that the large language model can understand the specific process and results of the code execution and make subsequent answer inferences based on the code execution results; the purpose of displaying the code execution results is to: verify the correctness of the code and ensure the accuracy of information extraction, especially for the information extraction task of WBS table data; assist the large language model in understanding the WBS table data and task breakdown structure, reducing the risk of hallucinations; and provide reliable data support for generating the final headcount estimation answer.
[0175] In step 5, the answer inference and generation step further includes: the large language model receives the natural language question, the natural language description of the WBS table structure, and the code execution results, and combines its pre-trained knowledge and reasoning ability to deeply understand and analyze the question, focusing on understanding the work content description proposed by the user and the relationship between tasks and number of people in the WBS table;
[0176] Based on its understanding of the problem and the data support provided by the code execution results, the large language model performs multi-step reasoning. This multi-step reasoning includes: analyzing the correlation between the code execution results and the natural language question to determine whether the code execution results can directly answer the headcount estimation question; if the code execution results cannot directly answer the headcount estimation question, further analyzing the deep semantics and potential information contained in the code execution results, such as analyzing the relationship between different task types, task complexity, and the required number of people; combining the natural language description of the WBS table structure to understand the impact of factors such as task hierarchy and dependencies on headcount estimation; and gradually deriving the final answer using a reasoning chain. The reasoning chain includes intermediate reasoning steps and logical deduction processes, such as "First, determine the task complexity based on the task type; then, refer to the headcount configuration of similar tasks in the WBS table; then, consider the dependencies and parallelism between tasks; and finally, based on all these factors, estimate the number of people required to complete the task." Finally, the reasoned answer is organized and expressed in natural language form to generate a final natural language answer. This natural language answer concisely, clearly, and accurately answers the work content proposed by the user and provides a reasonable headcount estimate.
[0177] In step 6, the table data preprocessing step is specifically as follows: when the original power project WBS table data contains a large number of rows or columns, or the token length of the table data exceeds the context window limit of the large language model, in order to ensure that the model can effectively process the table data and reduce computing resource consumption, the table data preprocessing is performed;
[0178] The table data preprocessing adopts an adaptive cell integration algorithm to compress the table data token length while retaining the key information in the table data as much as possible, especially the task information and resource information related to the headcount estimation;
[0179] The cell integration algorithm includes:
[0180] The strategy based on row expansion prioritizes the integration of cells in the same row, assuming that WBS task items in the same row have higher semantic relevance. The strategy based on column expansion prioritizes the integration of cells in the same column, assuming that cells in the same column describe the same type of attribute information, such as working hours or number of people.
[0181] The spiral expansion strategy is to expand and integrate cells in a spiral manner, taking into account information in both row and column directions to more comprehensively preserve the structure and content of the WBS table;
[0182] In step six, the cell integration algorithm dynamically adjusts the integration strategy and strength based on the preset token length threshold to ensure that the token length of the preprocessed table data meets the model input requirements while preserving the integrity and valid information of the table data to the greatest extent possible, providing support for the subsequent WBS table structure self-explanation, code generation, and headcount estimation answer inference steps.
[0183] In step 6, when the original WBS table data, after being converted into textual or structured representation according to the method in step 2, has a token length exceeding the context window limit of the large language model (e.g., common 4096, 8192, 16K or higher tokens, depending on the selected large language model), or in order to improve data processing efficiency and reduce computing resource consumption, the table data preprocessing step is used to intelligently compress and streamline the table data;
[0184] The core goal of tabular data preprocessing is to ensure that, while strictly adhering to the model's token budget, a series of strategic operations are performed to ensure that the data input to the large language model retains as much information as possible that is critical for answering user questions and completing subsequent reasoning tasks (such as headcount estimation), while effectively avoiding the loss of key contextual information caused by simple data truncation.
[0185] Methods and strategies for preprocessing tabular data include:
[0186] (1) Intelligent Filtering and Pruning, specifically:
[0187] a. Dynamic filtering based on question relevance: Analyze the semantics of the user's natural language questions to identify core entities, intent, and scope. Based on this, dynamically filter out rows (task items), columns (attribute fields), or specific data subsets in the WBS table that are directly related to the question topic. For example, if a user asks about the number of people in a work package at a specific project phase, the system will prioritize extracting relevant data columns for this work package and its direct subtasks within that project phase (such as task ID, task name, planned hours, required number of people, predecessor tasks, etc.).
[0188] b. Removal of structurally irrelevant information: Systematically remove columns in the table that contribute little or are irrelevant to the core WBS analysis (task decomposition, time estimation, resource allocation, dependency identification, etc.), such as format control characters, non-business-related comment columns, columns with a high proportion of null values, or auxiliary information columns that do not meet the current analysis objectives.
[0189] (2) Data Aggregation and Summarization, specifically:
[0190] a. Hierarchical aggregation of numerical data: For WBS tables with a clear hierarchical structure (e.g., work package-task-subtask), numerical data (e.g., planned hours, actual hours, required headcount) from sub-tasks can be meaningfully aggregated (e.g., sum, weighted average, maximum / minimum values, etc.) to parent tasks. This significantly reduces the number of data entries while preserving macro-trend and aggregate information. For example, summing the "required headcount" of all subtasks can yield the "total required headcount" of their immediate parent task.
[0191] b. Descriptive text summary: For cells containing long descriptive text (such as detailed task descriptions, notes, etc.), a lightweight text summarization algorithm can be used or specially designed prompts can be used to guide the large language model itself (possibly using a model with a smaller context window or block summary) to generate a short summary of its core content, thereby significantly reducing its token usage.
[0192] (3) Chunking and Iterative Information Integration, specifically:
[0193] a. Logical chunking or token budget chunking: Split large table data into multiple independent but potentially logically related data chunks or sub-tables based on their inherent logical structure (e.g., by major work packages, by project phases, by major dependency chains) or a preset token limit (e.g., a value slightly less than the model's maximum token limit);
[0194] b. Block processing and result fusion: Each data block is fed into the large language model for analysis, extracting key information, forming preliminary judgments, or calculating intermediate results. Subsequently, a result integration and comprehensive reasoning mechanism is designed (this mechanism itself can also leverage the reasoning capabilities of the large language model) to aggregate, compare, disambiguate, correlate, and comprehensively reason the analysis results from different data blocks to form a comprehensive understanding of the entire original table and the final answer. For example, when estimating the total number of people, the estimated number of people in each block is summed up or more complex dependency adjustments are performed.
[0195] (4) Sliding Window Approach, specifically:
[0196] When the table data is completely serialized into a long text (such as converted to Markdown or JSON string), the sliding window technique can be used. A window of fixed size (in tokens) is set, which slides gradually over the serialized text. The model processes the text fragment within the current window at each window position. To maintain contextual coherence, a certain overlapping area can be set between adjacent windows. The model processes the content of each window and gradually accumulates or transmits the processing results (such as extracted information and intermediate conclusions) to build an understanding of the entire table content.
[0197] (5) Optimized cell integration and information density maintenance strategies, specifically:
[0198] Apply the cell integration algorithm based on row expansion, column expansion or spiral expansion in step 6; during the integration process, introduce an information value or information density assessment mechanism; for example, prioritize retaining cell contents containing key numerical data, specific business keywords, or those judged to be more relevant based on the user's question, while cell contents with high repetition or low information content may be appropriately compressed or omitted during the integration. (6) Embedding-based contextual retrieval augmentation, specifically: semantically embed the rows of the table, meaningful cell combinations or pre-defined sub-tables (data blocks), and store these embedding vectors and their corresponding original data fragments in the vector database. When the user asks a question, the user's question is first converted into an embedding vector, and then the most relevant table data fragments are found in the vector database through similarity search. These retrieved, highly relevant fragments are then combined (within the token limit) as contextual input for the large language model to perform final reasoning and answer.
[0199] The above preprocessing method enhances the processing capability, accuracy, and robustness of WBS tables for power projects containing massive amounts of data and extremely complex structures, ensuring that key information can be effectively extracted and utilized even under strict token length restrictions, providing users with valuable analysis results and problem answers.
[0200] The method further includes step seven, performing a model performance evaluation, namely, evaluating the performance of the method, including the accuracy of the answers and their relevance to natural language questions, as well as the efficiency of code generation and execution; the evaluation is performed using a standard test dataset containing power project WBS table data, user task descriptions, and headcount estimation results;
[0201] The model performance evaluation in step seven is specifically as follows: In order to comprehensively and objectively evaluate the effectiveness of the method in the WBS table data reasoning and question-answering tasks of power projects, an in-depth evaluation is conducted from multiple dimensions using a typical application scenario (headcount estimation);
[0202] First, regarding answer accuracy, for key tasks like headcount estimation, the accuracy of headcount estimation results will be used as a core metric, combined with manual evaluation to comprehensively measure the quality of the answers generated by the model. The headcount estimation accuracy focuses on assessing the degree of consistency between the model's predicted headcount for this type of task and the actual headcount or expert estimates in the WBS. Meanwhile, manual evaluation focuses on determining the rationality and feasibility of various answers, including headcount estimation, from the perspective of project management and resource allocation, thus compensating for the shortcomings of automated evaluation metrics in terms of domain knowledge and practical experience.
[0203] Secondly, in terms of answer relevance, we will evaluate whether the natural language answers generated by the model can effectively answer users' questions about work content and headcount estimation, whether the answers are relevant, whether they include key information of concern to users such as task breakdown structure, time estimates, resource requirements, and whether the answers are expressed clearly, fluently, and easily understood. Relevance is evaluated manually, with project management experts or senior engineers scoring the answers generated by the model and evaluating them based on multiple dimensions, such as the completeness of the answers, the amount of information, practicality, and the naturalness of the language.
[0204] In terms of code generation and execution efficiency, the focus will be on evaluating the quality of the model-generated code, including its executable nature, logical correctness, and efficiency, particularly its ability to accurately extract task information from the WBS table and perform calculations related to headcount estimation. The time required for code execution and the response speed of the entire question-and-answer system will also be evaluated to measure the efficiency and real-time performance of the proposed method in practical applications. Efficiency evaluation will be conducted using automated testing and performance analysis tools, coupled with stress testing in conjunction with actual application scenarios, to fully understand the performance bottlenecks and optimization potential of the adopted model.
[0205] In terms of text classification accuracy evaluation, the confusion matrix is used to record the performance of the model, including precision P, recall R and F1 value as evaluation indicators, which are calculated through the confusion matrix.
[0206] The confusion matrix of the classification results is expressed in a table as follows:
[0207]
[0208] P refers to the proportion of samples predicted as positive by the classifier and correctly predicted to all samples predicted as positive. The calculation formula is as follows:
[0209]
[0210] R refers to the proportion of samples predicted as positive by the classifier and predicted correctly to all truly positive samples. The calculation formula is as follows:
[0211]
[0212] F1 is a comprehensive indicator of P and R, and its calculation formula is as follows:
[0213]
[0214] In terms of answer accuracy evaluation, for the headcount estimation task, the headcount estimation accuracy and exact match rate (EM) will be used as evaluation indicators. The headcount estimation accuracy focuses on evaluating the degree of consistency between the number of people predicted by the model and the actual number of people or the expert estimate in the WBS table. The root mean square error (RMSE) or mean absolute percentage error (MAPE) indicators are used to quantify the deviation of the headcount estimation. Lower RMSE and MAPE values indicate higher headcount estimation accuracy.
[0215] For non-numeric answers, such as task names and resource types, the exact match rate evaluation model is used to return the proportion of answers that are completely consistent with the standard answer. The higher the EM value, the more accurate the answer.
[0216] To more closely analyze the types of model errors, we roughly categorized the questions by question type (e.g., headcount estimation, task information query, etc.). For each type of question, we analyzed the model's prediction results, counting the true positives (TP) and false negatives (FN). We also calculated precision, recall, and F1-score to help analyze the model's performance on different types of questions.
[0217] This example addresses reasoning and question answering about power project WBS tables. Accurately estimating the resources (such as manpower) required for a task is a crucial application. Therefore, the evaluation will focus specifically on the accuracy of tasks such as headcount estimation, the relevance of answers, and the efficiency of code generation and execution. The evaluation method will combine automated and manual evaluation, and perform comparative performance analysis using a standard test dataset.
[0218] In this example, the accuracy of answers will be assessed using the headcount estimation task, a common problem in power project management. Headcount estimation accuracy and exact match (EM) will be used as evaluation metrics. Headcount estimation accuracy focuses on assessing the degree of consistency between the model's predicted headcount and the actual or expert estimates in the WBS. Metrics such as the root mean square error (RMSE) or mean absolute percentage error (MAPE) can be used to quantify the deviation in headcount estimation. Lower RMSE and MAPE values indicate higher headcount estimation accuracy. For other types of answers, such as non-numeric answers like task names and resource types, the exact match rate is used to assess the proportion of answers returned by the model that are completely consistent with the standard answer. Higher EM values indicate higher answer accuracy. To more closely analyze the types of model errors, this example draws on the concept of a confusion matrix. Although the method described in this example primarily addresses reasoning and question-answering tasks, rather than pure classification tasks, the concept of a confusion matrix can help understand the model's performance on different types of questions. For example, questions can be roughly categorized by question type (e.g., headcount estimation, task information query, etc.). Then, for each type of question, the model's prediction results can be analyzed, with metrics such as true positives (TP) and false negatives (FN) counted. Precision, recall, and F1-score can also be calculated to help analyze the model's performance on different types of questions. Note that the Precision, Recall, and F1-score metrics used here are inspired by the confusion matrix and are not strictly text classification metrics. Instead, they assist in analyzing the model's performance on different types of questions, focusing on analyzing error types rather than simply pursuing numerical classification metrics.
[0219] To ensure the objectivity and comparability of the evaluation results, a standard test dataset containing power project WBS table data, various types of natural language questions (including but not limited to task descriptions and headcount estimation requests), and corresponding standard answers (such as headcount estimation results) will be selected as the test benchmark. Furthermore, an evaluation dataset containing natural language questions of varying types and difficulty levels regarding work content and headcount estimation will be constructed to comprehensively examine the adaptability and robustness of the model in different scenarios.
[0220] When evaluating the results, the method will be compared with existing advanced table data analysis and project management tools, such as traditional spreadsheet software, project management software, and other artificial intelligence-based resource estimation models, to fully demonstrate the advantages and innovations of the method in WBS table data reasoning and headcount estimation.
[0221] The large language model is DeepSeek V3.
Claims
1. A method for reasoning and question-answering tabular data in the power grid domain based on a code-enhanced framework, characterized by: The proposed method aims to understand WBS table data and quickly estimate human resource requirements under zero-sample conditions. By constructing a code enhancement framework and utilizing a large language model to handle reasoning and question-answering tasks for WBS table data of power projects, the method first performs structural self-explanation on the WBS table data of power projects to improve the model's ability to understand the complex WBS table hierarchical structure and task decomposition relationships. Then, based on the table structure description and natural language questions formed by user-proposed work content descriptions, the method automatically generates executable code to extract relevant task information and resource allocation data from the WBS table data. The code is then executed through an external code interpreter to obtain results. Finally, the code execution results are combined with the model's own reasoning ability to generate natural language answers about the number of human resources required to complete specific work content.
2. The method for reasoning and question-answering tabular data in the power grid field based on a code-enhanced framework according to claim 1, characterized in that: The method comprises the following steps: Step 1: Question Receive and Table Data Preparation. Specifically, the process involves receiving a natural language question about the power project WBS table data, which is a user-proposed work content description. The WBS table data containing multi-level headers is then obtained. The WBS table data must meet pre-set normalization requirements to ensure data consistency and parsability. Step 2: Self-explanatory table structure; specifically: First, in order to enable the large language model to effectively process table data, the WBS table data that meets the standardization requirements is converted into a textual or structured representation that is easy for the large language model to understand and process, including serializing it into a CSV string, Markdown table format, JSON object, or simplified HTML table code format. The format to be used can be flexibly selected according to the amount of data processing. Then, the large language model is used to perform structural analysis on this converted table data representation. The core of this step is to guide the model to generate a rich and structured natural language description of the table through prompt word engineering; Step 3: Code generation and information extraction. Specifically, based on the natural language question and the natural language description of the WBS table structure, the large language model generates code for information extraction. The code is used to extract rows and columns related to the work content proposed by the user from the WBS table data, and perform data filtering based on the WBS table structure and user requirements. Data filtering includes filtering tasks under specific work packages or aggregating and calculating the total working hours of a certain type of tasks. Step 4: Code execution and result display; specifically, executing the generated code using an external code interpreter to obtain code execution results; displaying the code execution results in natural language to facilitate understanding and utilization by the large language model; the external code interpreter is a Python interpreter or an SQL execution engine; Step 5: Answer inference and generation: Based on the natural language question, the natural language description of the WBS table structure, and the code execution results, the large language model infers the answer. Utilizing the large language model's reasoning capabilities, combined with numerical information such as working hours and number of people in the WBS table data, as well as semantic information such as task hierarchy and dependencies, the model infers and generates an estimate of the number of people required to complete the task. The estimate is presented as a natural language answer. Step 6: Preprocessing the table data. Specifically, if the WBS table data volume is too large or the model input token length is limited, preprocess the table data in steps 1 and 2. The table data preprocessing method includes: a) Identify the header rows and columns of the WBS table and extract the required header information; b) Based on the header information and the work content proposed by the user, filter out the relevant WBS task items; c) Integrate the filtered table data with the relevant cells in the original table data to maximize the retention of valid information; d) Control the token length of the pre-processed table data within the preset threshold range; The table data preprocessing method includes: a cell integration method based on row expansion, column expansion or spiral expansion strategy.
3. The method for reasoning and question-answering tabular data in the power grid field based on a code-enhanced framework according to claim 2, characterized in that: In step 1, the preset normalization limits include: Standardize table header information, including unified naming conventions and clear structure hierarchy; Use clear methods to represent task hierarchical relationships, including using WBS codes or parent-child task IDs for identification; Standardize the units of key numerical data and the format of text data; key numerical data include working hours and resource quantities; the tabular data includes information such as the project work breakdown structure, working hour estimates, and required number of people; In step 1, the grid domain table data is the power project WBS table data, which includes the following types of data: Basic project information, including project name, project number, project cycle, and project participants; Work breakdown structure information, which records the hierarchical structure of the project's work packages, work tasks, and subtasks, as well as the task names, descriptions, and work content; Man-hour estimation information, which records the planned man-hours, actual man-hours, and remaining man-hours for each work task or subtask, is used for schedule management and control; Human resource information, which records the positions, skill requirements, and number of personnel required to complete each work task or subtask, for use in human resource allocation and cost estimation; Task dependency information, which records the sequence, dependencies, and constraints between work tasks, is used for project schedule planning and optimization; Project cost information, which records the cost budget, actual cost, deviation analysis, and other information for each work task or subtask, for project cost control and performance evaluation; The natural language questions are replaced by natural language questions raised by users regarding work content and headcount estimation in the WBS table of the power project, including: headcount estimation, working hour query and estimation, resource allocation optimization, cost-benefit analysis, and risk response measure query.
4. The method for reasoning and question-answering tabular data in the power grid field based on a code-enhanced framework according to claim 3 is characterized by: In step 2, the prompt word project specifically includes the following guidance strategies and contents: a) Guidance on identifying the overall information of the table: prompting the model to first identify and describe the overall theme or title of the table; b) Multi-level header structure analysis guidance: For tables containing multi-level headers, the model is prompted to identify the header content level by level and describe its hierarchical relationship and coverage; c) Key column semantic understanding guidance: Prompts the model to identify and interpret the business meaning of key data columns, especially those related to task breakdown, working hours, number of people, and dependencies; d) Guidance on the correspondence between data cells and table headers: Prompts the model to understand how data rows correspond to the structure defined in the table header, ensuring that the model can associate specific data with its attributes. e) Structured output format guidance: requires the model to output information according to a predefined structure or template; The prompt words in the prompt word project guide the model to output natural language descriptions that include information about table header hierarchies, parent-child relationships, and key column semantics. These descriptions are structured and standardized, making them easier to understand and utilize in subsequent steps. In order to further improve the efficiency and accuracy of subsequent processing, the natural language description output by the large language model is subjected to structured parsing. Specifically, standardized table metadata is extracted from the natural language description of the model using keyword extraction, regular expression matching, or a rule-based parser. The table metadata is structured table metadata, including: a list of table headers and their hierarchy, clear identification of each key column and its semantic label, and a representation of the task hierarchy relationship; it is used to directly serve the subsequent code generation and answer inference steps; to ensure the accuracy and deconstructibility of the description, the correctness of the parser's parsing results is confirmed through rule verification, comparison with known templates, comparison and verification of its output with predefined standard data structure patterns, evaluation of its support effect on downstream task performance, or manual sampling review, thereby ensuring the effectiveness of subsequent processing based on this description; In step 2, the table structure self-explanation step further includes: the large language model first identifies the title of the WBS table, understands that the table is about the theme and content summary of the power project work breakdown structure and resource estimation; then, identifies the table headers, including row headers and column headers, and understands the meaning and data dimensions represented by each header; further, parses the hierarchical relationship of the multi-level headers of the WBS table to construct a WBS task hierarchy tree structure; then, analyzes the correspondence between the headers and data cells, and determines the header category and dimension to which each data cell belongs; finally, based on the above parsing results, generates a natural language description of the WBS table structure; The natural language description includes the hierarchical header structure of the table, the meaning of each header, and the semantic association between the header and the data cells. It is used to assist the large language model in understanding the organization and intrinsic meaning of the tabular data, improving the accuracy of subsequent information extraction and reasoning, and especially assisting in understanding the relationship between the table structure and the task of estimating the number of people.
5. The method for reasoning and question-answering tabular data in the power grid field based on a code-enhanced framework according to claim 3 is characterized by: In step three, the code generation and information extraction steps include: determining the code type to be executed using a large language model based on the intent of the natural language question and the natural language description of the WBS table structure, wherein the code type includes but is not limited to Python scripts or SQL query statements; based on the determined code type, the large language model generates specific code instructions, wherein the code instructions include selection statements for locating relevant work packages, work tasks, or subtasks in the WBS table data, and functions or methods for performing data operations; the data operations are replaced with operations related to the WBS table data, including: Task filtering: filtering WBS tasks based on specific conditions. Task filtering includes filtering all tasks under a specific work package or filtering tasks on the critical path. Task sorting: sort WBS tasks according to one or more columns of data; Data aggregation, statistical calculation of WBS table data; Task dependency analysis analyzes the dependencies between WBS tasks; the generated code is designed to extract the information needed to answer natural language questions about work content and headcount estimation from WBS table data, and organize the extracted information into structured data to facilitate subsequent answer inference and generation for headcount estimation reasoning.
6. The method for reasoning and question-answering tabular data in the power grid field based on a code-enhanced framework according to claim 3, characterized in that: Step 4 includes code execution and result display, which includes: using an external Python interpreter or SQL execution engine to execute the code generated in step 3, obtaining the result after code execution, which is a structured WBS table data subset or value; converting the code execution result into a natural language description, which is a summary and explanation of the code execution result; The code execution results and natural language descriptions are simultaneously presented to the large language model so that the large language model can understand the specific process and results of the code execution and make subsequent answer inferences based on the code execution results. The purpose of presenting the code execution results is to: verify the correctness of the code and ensure the accuracy of information extraction, especially for the information extraction task of WBS table data; assist the large language model in understanding the WBS table data and task decomposition structure, reducing the risk of hallucinations; and provide reliable data support for generating the final headcount estimation answer.
7. The method for reasoning and question-answering tabular data in the power grid field based on a code-enhanced framework according to claim 3, characterized in that: In step 5, the answer inference and generation step further includes: the large language model receives the natural language question, the natural language description of the WBS table structure, and the code execution results, and combines its pre-trained knowledge and reasoning ability to deeply understand and analyze the question, focusing on understanding the work content description proposed by the user and the relationship between tasks and number of people in the WBS table; Based on its understanding of the problem and the data support provided by the code execution results, the large language model performs multi-step reasoning. The multi-step reasoning includes: analyzing the correlation between the code execution results and the natural language question to determine whether the code execution results can directly answer the headcount estimation question; if the code execution results cannot directly answer the headcount estimation question, further analyzing the deep semantics and potential information contained in the code execution results; combining the natural language description of the WBS table structure to understand the impact of factors such as task hierarchy and dependencies on headcount estimation; using a reasoning chain that includes intermediate reasoning steps and a logical deduction process to gradually derive the final answer; and finally, organizing and expressing the reasoned answer in natural language to generate the final natural language answer.
8. The method for reasoning and question-answering tabular data in the power grid field based on a code-enhanced framework according to claim 3, characterized in that: In step 6, the table data preprocessing step is specifically as follows: when the original power project WBS table data contains a large number of rows or columns, or the token length of the table data exceeds the context window limit of the large language model, in order to ensure that the model can effectively process the table data and reduce computing resource consumption, the table data preprocessing is performed; The table data preprocessing adopts an adaptive cell integration algorithm to compress the table data token length while retaining the key information in the table data as much as possible, especially the task information and resource information related to the headcount estimation; The cell integration algorithm includes: The strategy based on row expansion is to prioritize the integration of cells in the same row, assuming that WBS task items in the same row have higher semantic relevance. The strategy based on column expansion is to prioritize the integration of cells in the same column, assuming that cells in the same column describe the same type of attribute information. The spiral expansion strategy is to expand and integrate cells in a spiral manner, taking into account information in both row and column directions to more comprehensively preserve the structure and content of the WBS table; In step six, the cell integration algorithm dynamically adjusts the integration strategy and strength based on the preset token length threshold to ensure that the token length of the preprocessed table data meets the model input requirements while preserving the integrity and valid information of the table data to the greatest extent possible, providing support for the subsequent WBS table structure self-explanation, code generation, and headcount estimation answer inference steps. In step 6, when the original WBS table data, after being converted into textual or structured representation according to the method in step 2, has a token length exceeding the context window limit of the large language model, or in order to improve data processing efficiency and reduce computing resource consumption, the table data preprocessing step is used to intelligently compress and streamline the table data; The core goal of tabular data preprocessing is to ensure that, while strictly adhering to the model's token budget, a series of strategic operations are performed to ensure that the data input to the large language model retains as much information as possible that is important for answering user questions and completing subsequent reasoning tasks, while avoiding the loss of key contextual information caused by simple data truncation. Methods and strategies for preprocessing tabular data include: (1) Intelligent data screening and pruning, specifically: a. Dynamic filtering based on question relevance: Analyze the semantics of the user's natural language questions to identify core entities, intent, and scope. Based on this, dynamically filter out rows, columns, or specific data subsets in the WBS table that are directly related to the question topic. b. Removal of structurally irrelevant information: Systematically remove columns in the table that have little contribution or are irrelevant to the core WBS analysis; (2) Data aggregation and summarization, specifically: a. Hierarchical aggregation of numerical data: For WBS tables with a clear hierarchical structure, the numerical data of sub-level tasks are meaningfully aggregated to the upper-level tasks, thereby reducing the number of data entries while retaining macro-trend and total information; b. Descriptive text summary: For cells containing long descriptive text, a lightweight text summarization algorithm or specially designed prompts are used to guide the large language model to generate a brief summary of its core content to reduce its token usage. (3) Block processing and iterative information integration, specifically: a. Logical or token-budget segmentation: Split large table data into multiple independent but potentially logically related data blocks or sub-tables based on their inherent logical structure or the preset token limit. b. Block processing and result fusion: Each data block is fed into the large language model for analysis, extracting key information, forming preliminary judgments, or calculating intermediate results. Subsequently, a result integration and comprehensive reasoning mechanism is designed to aggregate, compare, disambiguate, correlate, and perform comprehensive reasoning on the analysis results from different data blocks to form a comprehensive understanding of the entire original table and the final answer. (4) Sliding window mechanism: Specifically, when the table data is completely serialized into a long text, a sliding window technique is used. A fixed-size window, measured in tokens, is set and gradually slides over the serialized text. The model processes the text fragment within each window position. To maintain contextual coherence, a certain overlap area is set between adjacent windows. The model processes the content of each window and gradually accumulates or transmits the processing results to build an understanding of the entire table content. (5) Optimized cell integration and information density maintenance strategies, specifically: Apply the cell integration algorithm based on row expansion, column expansion, or spiral expansion in step 6; during the integration process, introduce an information value or information density evaluation mechanism; (6) Embedding-based contextual retrieval enhancement, specifically: Semantic embedding is performed on table rows, meaningful cell groups, or predefined subtables, and these embedding vectors and their corresponding original data segments are stored in a vector database. When a user asks a question, the question is first converted into an embedding vector. A similarity search is then performed in the vector database to identify the most relevant table data segments. These retrieved, highly relevant segments are then combined, within token limits, and used as contextual input for the large language model's final inference and answer.
9. The method for reasoning and question-answering tabular data in the power grid field based on a code-enhanced framework according to claim 3, characterized in that: The method further includes step seven, performing a model performance evaluation, namely, evaluating the performance of the method, including the accuracy of the answers and their relevance to natural language questions, as well as the efficiency of code generation and execution; the evaluation is performed using a standard test dataset containing power project WBS table data, user task descriptions, and headcount estimation results; The model performance evaluation in step seven is specifically as follows: In order to comprehensively and objectively evaluate the effectiveness of the method in the WBS table data reasoning and question-answering tasks of power projects, an in-depth evaluation is conducted from multiple dimensions using typical application scenarios; First, in terms of answer accuracy, for key tasks such as headcount estimation, we use the accuracy of headcount estimation results as a core metric, combined with manual evaluation, to measure the quality of the answers generated by the model. The accuracy of headcount estimation focuses on evaluating the degree of consistency between the model's predicted headcount for this type of task and the actual headcount or expert estimates in the WBS. Meanwhile, manual evaluation focuses on judging the rationality and feasibility of various answers, including headcount estimates, from the perspective of project management and resource allocation, thus compensating for the shortcomings of automated evaluation indicators in terms of domain knowledge and practical experience. Second, in terms of answer relevance, we will evaluate whether the natural language answers generated by the model can effectively answer users' questions about work content and headcount estimation, whether the answers are relevant, whether they include key information that users care about, such as task breakdown structure, time estimates, and resource requirements, and whether the answers are expressed clearly and fluently, making them easy to understand. Relevance assessment uses manual evaluation, inviting project management experts or senior engineers to score the answers generated by the model and evaluate them from multiple dimensions; In terms of code generation and execution efficiency, the focus is on evaluating the quality of the model-generated code, including its executable nature, logical correctness, and efficiency. Specifically, the code's ability to accurately extract task information from the WBS and perform calculations related to headcount estimation will also be evaluated. The time it takes to execute the code and the response speed of the entire question-answering system will also be assessed to measure the efficiency and real-time nature of this method in practical applications. Efficiency evaluation is conducted through automated testing and performance analysis tools, and stress testing is performed in combination with actual application scenarios to fully understand the performance bottlenecks and optimization space of the model used in this method; In terms of text classification accuracy evaluation, the confusion matrix is used to record the performance of the model, including precision P, recall R and F1 value as evaluation indicators, which are calculated through the confusion matrix. The confusion matrix of the classification results is expressed in a table as follows: P refers to the proportion of samples predicted as positive by the classifier and correctly predicted to all samples predicted as positive. The calculation formula is as follows: R refers to the proportion of samples predicted as positive by the classifier and predicted correctly to all truly positive samples. The calculation formula is as follows: F1 is a comprehensive indicator of P and R, and its calculation formula is as follows: In terms of answer accuracy evaluation, for the headcount estimation task, the headcount estimation accuracy and exact match rate (EM) will be used as evaluation indicators. The headcount estimation accuracy focuses on evaluating the degree of consistency between the number of people predicted by the model and the actual number of people or the expert estimate in the WBS table. The root mean square error (RMSE) or mean absolute percentage error (MAPE) indicators are used to quantify the deviation of the headcount estimation. Lower RMSE and MAPE values indicate higher headcount estimation accuracy. For non-numerical answers, the exact match rate is used for evaluation, which is the proportion of answers returned by the model that are completely consistent with the standard answer. The higher the EM value, the more accurate the answer. To more closely analyze the types of model errors, we roughly categorized the questions by question type. For each type of question, we analyzed the model's prediction results, calculated the true positives (TP) and false negatives (FN) metrics, and calculated the precision, recall, and F1 values to assist in analyzing the model's performance on different types of questions. To ensure the objectivity and comparability of the evaluation results, a standard test dataset containing WBS table data of power projects, various types of natural language questions and corresponding standard answers will be selected as the test benchmark. An evaluation dataset containing natural language questions of different types and levels of difficulty regarding work content and headcount estimation will be constructed to comprehensively examine the adaptability and robustness of the model in different scenarios.
10. The method for reasoning and question-answering tabular data in the power grid field based on a code-enhanced framework according to claim 3, characterized in that: The large language model is DeepSeekV3.
Citation Information
Cited By
File identification processing system based on artificial intelligence model and RAG
CN120894793A
Automatic code generation method for dynamic intelligent calculation and interactive analysis of power grid
CN121411755A
An excel data knowledge question and answer method based on table structure semantic cooperation
CN122390090A
An excel data knowledge question and answer method based on table structure semantic cooperation
CN122390090B