Spreadsheet standardization processing method and system based on large language model

Abstract

The application provides a large language model-based electronic form standardization processing method and system, and belongs to the technical field of data processing, and specifically comprises the following steps: a file acquisition module is responsible for intelligently screening a target worksheet set containing valid data; a slicing module is responsible for partitioning the content of the worksheet to obtain a plurality of data slices; the table header of each data slice and sample data are input into a large language model workflow to intelligently identify the dimension type and structural characteristics of the data; an execution module is responsible for generating management strategies and prompt words, calling a large language model to dynamically generate a targeted data processing function code, executing the generated function code through a function execution engine, and standardizing and converting the data slices; and all successfully processed data is combined and exported as a standardized electronic form, thereby improving the efficiency of the standardization processing of the electronic form.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and in particular relates to a method and system for standardizing spreadsheet processing based on a large language model. Background Technology

[0002] In industries such as logistics, supply chain, and cross-border e-commerce, supplier quotations are often in the form of unstructured or semi-structured Excel files. These files vary in format and have complex layouts, leading to the following systemic challenges in data processing:

[0003] The lack of standardized formats and the significant differences in table layout, column definitions, and data organization among different suppliers make it difficult to directly integrate and compare cross-source data.

[0004] The structure is highly complex, with a single quotation often containing multiple logistics methods and data presented in various formats (such as column merging and row merging), making it difficult for traditional parsing methods to adapt to this dynamic structure.

[0005] Manual processing is costly. Currently, it relies on manual identification, understanding and conversion of quotation formats one by one, which is inefficient and has a high error rate, and cannot meet the needs of large-scale and real-time business.

[0006] The rules lack generalization ability. Data processing tools based on predefined rules or fixed templates are difficult to cover the ever-changing file formats. The rules have high maintenance costs and poor scalability.

[0007] The level of intelligence is low. Existing tools lack the ability to understand the semantics of table content, cannot dynamically adjust the parsing logic according to data characteristics, and do not have adaptive and self-learning capabilities.

[0008] While some Excel data processing tools exist on the market, their core functionality still relies on fixed rules or manually configured templates, making them ineffective in handling unstructured and diverse spreadsheet data. Traditional methods have significant shortcomings in terms of flexibility, intelligence, and automation, necessitating an innovative solution that integrates artificial intelligence technology and possesses semantic understanding and adaptive parsing capabilities.

[0009] Therefore, there is an urgent need for a standardized processing method and system for spreadsheets based on a large language model. Summary of the Invention

[0010] To achieve the objectives of this invention, the following technical solution is adopted:

[0011] Specifically, this application provides a spreadsheet standardization processing system based on a large language model, which includes:

[0012] The file acquisition module is responsible for obtaining the file path and basic information of the spreadsheet.

[0013] The structure parsing module extracts a list of names of all worksheets in the spreadsheet to obtain the type of the spreadsheet. Based on the historical extraction deviation data of the type of the spreadsheet, when it is determined that the spreadsheet does not belong to the manually extracted matching table, the name list is input into the big language model workflow. Combined with domain keywords, semantic relevance analysis is performed on the worksheets to intelligently filter out the target worksheet set containing valid data.

[0014] The slicing module is responsible for partitioning the worksheet content to obtain multiple data slices. The header and sample data of each data slice are input into the large language model workflow to intelligently identify the dimension type and structural features of the data.

[0015] The execution module is responsible for determining the generation and management strategy of prompt words for the spreadsheet based on the deviation between the extracted data and the spreadsheet of the specified type, as well as the historical extracted data of the spreadsheet of the specified type. Based on the generation and management strategy and the prompt words, it calls the large language model to dynamically generate targeted data processing function code, executes the generated function code through the function execution engine, performs standardization transformation on the data slices, and merges and exports all successfully processed data into a standardized spreadsheet.

[0016] Furthermore, the keywords in this field include logistics, pricing, and transportation.

[0017] Furthermore, the worksheet content includes cell data, merged cell information, and the original layout structure, which are specifically determined based on the parsing results of the valid data.

[0018] Furthermore, the worksheet content is partitioned, specifically including:

[0019] Automatic layout type detection determines whether a layout is horizontal or vertical by statistically analyzing the distribution characteristics of key identifiers in rows and columns.

[0020] Intelligent data boundary identification combines empty row / column detection, keyword matching, and header feature recognition to determine the boundaries of each data block;

[0021] Multi-pattern matching algorithm: It uses matching algorithms including exact matching, prefix matching, and inclusion matching to identify the location of data identifiers, and supports name variations and additional information.

[0022] Furthermore, the dimension types and structural features include dimension type identification: such as country dimension, warehouse dimension, postal code dimension, and weight dimension; key field identification: such as the position and meaning of price column, timeliness column, and remarks column.

[0023] Furthermore, after obtaining the data processing function code, the generated data processing function code is saved to the function library to support subsequent rapid reuse and version management.

[0024] A spreadsheet standardization method based on a large language model, applied to the aforementioned spreadsheet standardization system based on a large language model, specifically includes:

[0025] S1 uses the spreadsheet's recognition data as a basis to determine the type of the spreadsheet. Based on the historical extraction deviation data of the spreadsheet of the specified type, if it is determined that the spreadsheet does not belong to the manually extracted matching spreadsheet, proceed to the next step.

[0026] S2 determines the deviation between the extracted data of the spreadsheet and the spreadsheet of the type, and, in conjunction with the historical extracted data of the spreadsheet of the type, determines the generation management strategy for the prompt words of the spreadsheet. Based on the generation management strategy, the prompt words of the large language model are generated. According to the similarity of the extraction functions corresponding to different prompt words, the same extraction functions are divided into the same extraction function group. Based on the extraction function group data and the prompt words in different extraction function groups, the termination control strategy for the extracted prompt words of the spreadsheet is determined.

[0027] After S3 determines that the spreadsheet of the type has reached the stop control strategy, it extracts the matching status of the function with the generation management strategy. Based on the matching status of different spreadsheets and the generation data of prompt words, it determines the update method of the manually extracted matching spreadsheet.

[0028] Furthermore, the identification data of the spreadsheet includes the spreadsheet's title and keywords.

[0029] Furthermore, the type of the spreadsheet is determined based on the parsing results of the spreadsheet's header.

[0030] Furthermore, determining that the spreadsheet is not a manually extracted matching spreadsheet specifically includes:

[0031] Based on the historical extraction data of the spreadsheet of the aforementioned type, determine the historical extraction count and the average daily extraction count of the spreadsheet of the aforementioned type;

[0032] Based on the historical extraction deviation data, determine the number of times the extraction results of the spreadsheet of this type are abnormal;

[0033] Based on the historical number of times the spreadsheet of this type was extracted and the number of times the extraction results were abnormal, it was determined whether the spreadsheet belonged to the manually extracted matching spreadsheet.

[0034] The beneficial effects of this invention are as follows:

[0035] No predefined rules are required; it can automatically understand and parse various complex spreadsheet formats, dynamically generate processing functions based on data characteristics, and support function archiving and reuse.

[0036] Intelligent error diagnosis and code correction are achieved through the LLM feedback mechanism. It automatically recognizes horizontal and vertical layouts, adapts to complex table structures, reduces manual work of several hours to minutes, and outputs files in a uniform format with complete metadata records.

[0037] Establish a performance-driven dynamic optimization mechanism: The system upgrades the maintenance of manual lists from static rules to a dynamic optimization process based on real-time performance data (failure rate, cost), enabling resource allocation strategies to adapt to changes in automated processing capabilities.

[0038] Achieving multi-dimensional and refined decision-making: The decision-making process comprehensively considers processing quality (failure rate) and processing efficiency (cost), and distinguishes between systemic risks and local problems through hierarchical logic, ensuring that the decision is both comprehensive and accurate.

[0039] Optimize overall operating costs: By identifying the "high cost-medium risk" type and flexibly adjusting the threshold, the system seeks the optimal balance between "resource consumption for automated processing" and "cost of manual processing", which helps to reduce overall operating costs.

[0040] Improve the robustness and explainability of the processing flow: All decisions are based on quantifiable metrics and a clear rule tree, making the reasons "why a certain type of form needs to be transferred to manual processing" traceable and explainable, thus enhancing the transparency and credibility of the entire processing flow.

[0041] Other features and advantages will be set forth in the following description, and the objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0043] The above and other features and advantages of the present invention will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0044] Figure 1 This is a framework diagram of a spreadsheet standardization processing system based on a large language model;

[0045] Figure 2 This is a flowchart of a spreadsheet standardization processing method based on a large language model;

[0046] Figure 3 This is a flowchart for determining whether a spreadsheet belongs to a manually extracted matching table;

[0047] Figure 4 This is a flowchart illustrating the method for determining the management strategy for generating prompts in spreadsheets. Detailed Implementation

[0048] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification. Example

[0049] like Figure 1 As shown, this application provides a spreadsheet standardization processing system based on a large language model, specifically including:

[0050] The file acquisition module is responsible for obtaining the file path and basic information of the spreadsheet.

[0051] The structure parsing module extracts a list of names of all worksheets in the spreadsheet to obtain the type of the spreadsheet. Based on the historical extraction deviation data of the type of the spreadsheet, when it is determined that the spreadsheet does not belong to the manually extracted matching table, the name list is input into the big language model workflow. Combined with domain keywords, semantic relevance analysis is performed on the worksheets to intelligently filter out the target worksheet set containing valid data.

[0052] The slicing module is responsible for partitioning the worksheet content to obtain multiple data slices. The header and sample data of each data slice are input into the large language model workflow to intelligently identify the dimension type and structural features of the data.

[0053] The execution module is responsible for determining the generation and management strategy of prompt words for the spreadsheet based on the deviation between the extracted data and the spreadsheet of the specified type, as well as the historical extracted data of the spreadsheet of the specified type. Based on the generation and management strategy and the prompt words, it calls the large language model to dynamically generate targeted data processing function code, executes the generated function code through the function execution engine, performs standardization transformation on the data slices, and merges and exports all successfully processed data into a standardized spreadsheet.

[0054] Furthermore, the keywords in this field include logistics, pricing, and transportation.

[0055] Furthermore, the worksheet content includes cell data, merged cell information, and the original layout structure, which are specifically determined based on the parsing results of the valid data.

[0056] Furthermore, the worksheet content is partitioned, specifically including:

[0057] Automatic layout type detection determines whether a layout is horizontal or vertical by statistically analyzing the distribution characteristics of key identifiers in rows and columns.

[0058] Intelligent data boundary identification combines empty row / column detection, keyword matching, and header feature recognition to determine the boundaries of each data block;

[0059] Multi-pattern matching algorithm: It uses matching algorithms including exact matching, prefix matching, and inclusion matching to identify the location of data identifiers, and supports name variations and additional information.

[0060] Furthermore, the dimension types and structural features include dimension type identification: such as country dimension, warehouse dimension, postal code dimension, and weight dimension; key field identification: such as the position and meaning of price column, timeliness column, and remarks column.

[0061] Furthermore, after obtaining the data processing function code, the generated data processing function code is saved to the function library to support subsequent rapid reuse and version management.

[0062] In one possible specific embodiment:

[0063] 1. Application Scenario: A cross-border e-commerce platform, "KuaMaoTong," receives price quotations from over 50 logistics providers daily for intelligent price comparison and route recommendation. These Excel files vary in format and have complex internal structures.

[0064] 2. System Deployment and Model Configuration:

[0065] LLM service: Deployed on Dify / Azure AI Studio using OpenAI GPT-4 API or open source models (such as Qwen-Max), building three workflows.

[0066] Workflow preset prompts example:

[0067] Relevance Filtering: "You are a logistics data analysis expert. Please analyze the following list of Excel worksheet names and filter out the worksheets most likely to contain logistics freight quote data. Simply return the worksheet names, separated by commas. List: ['Cover', 'Terms', 'A Air', 'B Sea', 'Notes']"

[0068] Dimensional Analysis: "Analyze the header and sample data of the following table fragment to determine its main data organization dimensions (e.g., country, warehouse, postal code range, weight range), and identify key columns such as 'price', 'transit time', and 'notes'. Return JSON format: {"primary_dimension": "", "columns": {"price": "column name or range", "transit_time": "column name", "notes": "column name"}}."

[0069] Function generation: "You are a Python programming expert. Please write a function to transform the raw data rows into the target format based on the following data schema. Target format schema: {…}. Data characteristics: Dimension type is 'Warehouse Dimension', Price column is 'Price', Weight column is 'WeightRange'. Pay attention to handling merged cells and blank values."

[0070] 3. Step-by-step deduction of the processing procedure (taking "Air Freight from Country A" as an example):

[0071] Step S3: LLM Filtering, Enter: ['Cover', 'Quotation Description', 'Air Freight from Country A', 'Sea Freight from Country B', 'Description']

[0072] LLM analysis process: The model, based on semantic understanding, identifies that "air freight from country A" and "sea freight from country B" contain keywords related to the "transportation" domain and geographical information, and are strongly correlated with "logistics quotes." "Cover," "Quote Description," and "Explanation" are more text-based descriptions with low data density.

[0073] Export: Air freight from country A, sea freight from country B;

[0074] Adaptive data slicing:

[0075] Input: Raw data from the "Air Freight from Country A" worksheet (assuming a horizontal layout):

[0076] Table 1 Example of a logistics table

[0077]

[0078] Algorithm execution:

[0079] Layout detection: The scan found that "Ordinary Air Passenger" and "Extreme Air Passenger" appeared in the same row (row 1), which was determined to be a horizontal layout.

[0080] Boundary recognition: The first row is the identifier row, and two identifiers are recognized.

[0081] Scanning to the right, B1 is "Normal Empty Disk". Its data area extends to the right until it encounters the next identifier, "Rapid Empty Disk" (D1), i.e., columns B:D (but D1 is the starting column of the next identifier, so the actual data boundary of "Normal Empty Disk" is B1:D4, which needs to be corrected according to the header row 2. In fact, data block 1 is B2:D4, and block 2 is E2:G4).

[0082] Scan down to consecutive blank lines or the end of the data (line 4).

[0083] Output slices: Slice1: {Identifier: "Normal empty slice", Data area: B2:D4}; Slice2: {Identifier: "Very fast empty slice", Data area: E2:G4};

[0084] Dimensional Analysis and Dynamic Function Generation: LLM Input (using Slice1 as an example): Header: ["Warehouse Code", "Price", "Delivery Time"], Sample Data: [{"Warehouse Code": "LAX1", "Price": "$5.00", "Delivery Time": "5 days"}]

[0085] LLM analysis output (JSON):

[0086] json{

[0087] "primary_dimension": "fba_warehouse",

[0088] “columns”: {

[0089] "destination": "repository code"

[0090] “price”: “price”

[0091] "transit_time": The time limit or validity period.

[0092] }

[0093] }

[0094] Function generation:

[0095] LLM generates the following functions based on the "Warehouse Dimension" template and the analysis above:

[0096] def transform_slice1(row):

[0097] return {

[0098] "Logistics method": "Ordinary air freight"

[0099] "Destination Type": "FBA Warehouse"

[0100] "Destination": row["repository code"],

[0101] "Price": extract_numeric(row["price"]), # LLM Smart adds currency symbol handling.

[0102] "Timeliness": extract_days(row["Timeliness"]) # LLM intelligently adds text day extraction

[0103] # … Other standardized fields

[0104] }

[0105] Function execution and intelligent retries: First execution success: The function processes all lines without errors.

[0106] Hypothetical error scenario: In a supplier's quote, the "Timeliness" column might contain "5-7 business days" or "N / A". First execution failure: extract_days("5-7 business days") may cause a formatting error.

[0107] Smart Retry: Error Context Collection: {"error_type": "ValueError", "error_msg": "Number of days unable to resolve", "offending_value": "5-7 business days", "function_code": "..."}

[0108] Feedback LLM Regeneration: Appends the error context to the original function's generated prompt, requests correction, and retryes execution: The new function successfully processes all lines.

[0109] The results are exported and ultimately generated as a standard CSV / Excel file, containing columns such as: logistics method, destination type, destination, initial weight price, delivery time (days), data source file, and processing timestamp.

[0110] Complex error scenarios:

[0111] Problem: The weight column in the quotation is a range in string format, such as "10-20 kg". However, the function generated by LLM on the first run may only use split("-") to process it, without taking into account the unit "kg" and spaces, which causes the value extraction to fail.

[0112] The system's retry decision-making process (managed by the retry controller):

[0113] Strategy: Not only retry, but also select a strategy based on the error type.

[0114] Syntax errors (such as Python syntax errors): Directly request the LLM to fix the syntax.

[0115] Logical / data errors (such as type conversion errors mentioned above): Feed back the erroneous data sample and the expected output format to the LLM, requiring it to enhance the robustness of the function and its data cleaning capabilities.

[0116] Timeout error: Try simplifying the logic of the generator function or dividing the input data into chunks.

[0117] LLM feedback learning and correction, enhanced prompt: "The previous function failed when processing the 'weight range' column because the string contains the unit 'kg'. Please analyze the following error and sample to regenerate a more robust parsing function that always returns the lower and upper limits of the value in kilograms."

[0118] 4. Advanced strategies for the retry controller:

[0119] Exponential backoff: Wait a short time before retrying to avoid frequent calls to the LLM API.

[0120] Manual review threshold: After two consecutive failed retries of the same data slice, the error context, input data, and all attempted function code are packaged into a structured report, which is then submitted to the manual processing queue through the Web service module.

[0121] Successful Experience Archiving: After a successful retry, the new function code and the description of the problem it solves will be stored in the "Problem-Solution" knowledge base for quick matching and function recommendation for similar problems in the future, reducing LLM calls.

[0122] Example 2

[0123] like Figure 2 As shown, a spreadsheet standardization method based on a large language model is applied to the aforementioned spreadsheet standardization system based on a large language model, specifically including:

[0124] S1 uses the spreadsheet's recognition data as a basis to determine the type of the spreadsheet. Based on the historical extraction deviation data of the spreadsheet of the specified type, if it is determined that the spreadsheet does not belong to the manually extracted matching spreadsheet, proceed to the next step.

[0125] S2 determines the deviation between the extracted data of the spreadsheet and the spreadsheet of the type, and, in conjunction with the historical extracted data of the spreadsheet of the type, determines the generation management strategy for the prompt words of the spreadsheet. Based on the generation management strategy, the prompt words of the large language model are generated. According to the similarity of the extraction functions corresponding to different prompt words, the same extraction functions are divided into the same extraction function group. Based on the extraction function group data and the prompt words in different extraction function groups, the termination control strategy for the extracted prompt words of the spreadsheet is determined.

[0126] After S3 determines that the spreadsheet of the type has reached the stop control strategy, it extracts the matching status of the function with the generation management strategy. Based on the matching status of different spreadsheets and the generation data of prompt words, it determines the update method of the manually extracted matching spreadsheet.

[0127] Furthermore, the identification data of the spreadsheet includes the spreadsheet's title and keywords.

[0128] Furthermore, the type of the spreadsheet is determined based on the parsing results of the spreadsheet's header.

[0129] Specifically, such as Figure 3 As shown, determining that the spreadsheet is not a manually extracted matching spreadsheet specifically includes:

[0130] The core objective of this embodiment is to automatically identify and filter spreadsheet types that are more suitable (or necessary) for manual verification and extraction, thereby achieving optimized division of labor through "human-machine collaboration" in the workflow of batch spreadsheet standardization using large models. Its core logic lies in: by analyzing historical extraction records (including extraction frequency and accuracy of extraction results) for specific types of spreadsheets, the reliability and business importance of the automatic extraction model (such as a large model) for processing these types of spreadsheets are evaluated. For spreadsheet types with low extraction frequency, small data volume but high accuracy requirements, or high historical automatic extraction error rates (anomaly rates), the system will determine them as "manually extracted matching spreadsheets," and recommend introducing a manual step into the standardization process for these types of spreadsheets to ensure final data quality and avoid business risks caused by model mis-extraction. Conversely, for high-frequency, stable, and automatically extracted accurate spreadsheet types, the process is entirely automated, improving overall efficiency.

[0131] S11 determines the historical extraction count and daily average extraction count of the spreadsheet based on the historical extraction data of the spreadsheet of the described type;

[0132] "Historical Extraction Data" refers to all records of this type of spreadsheet processed by the system (or manually) within a past period. Key fields include extraction time, extraction result, and verification result (whether it is correct). "Historical Extraction Count" is the total number of times this type of spreadsheet has been processed. "Daily Average Extraction Count" is the average of the historical extraction count divided by the number of days in the statistics, reflecting the processing frequency of this type of spreadsheet.

[0133] This step quantifies processing popularity and business participation. The number of extractions directly reflects the frequency and importance of this type of table in the business process. Frequently processed tables are often part of core business processes, and their processing efficiency directly impacts overall operational efficiency. The significance of this step is to provide objective data on "business volume" for subsequent judgments, avoiding decisions based solely on subjective impressions. Even if frequently processed tables have a certain error rate, a strategy of primarily automation supplemented by manual sampling may be adopted due to efficiency priorities.

[0134] S12 Based on the historical extraction deviation data, determine the number of times the extraction results of the spreadsheet of the type are abnormal;

[0135] "Historical extraction deviation data" specifically refers to historical extraction records marked as "abnormal" or "error". "Number of times extraction results were abnormal" refers to the number of cases within the statistical period where the automatically extracted results did not match the actual data in the table, which had been manually verified. This is a core indicator for measuring the accuracy of the automatic extraction model in processing this type of table.

[0136] This step quantifies the risks and costs of errors. The number of anomalies directly reveals the weaknesses or blind spots of the automatic extraction model when processing specific types of tables. Low model recognition rates may be due to complex, non-standard table structures, or unique layouts of key information. Its significance lies in transforming the abstract concept of "model unreliability" into concrete, statistically comparable numbers (number of anomalies), laying the foundation for subsequent calculations of the error rate (probability of deviation), and serving as one of the most crucial bases for determining whether manual intervention is necessary.

[0137] S13 determines whether the spreadsheet belongs to the manually extracted matching spreadsheet based on the historical number of times the spreadsheet of the described type has been extracted and the number of times the extraction results are abnormal.

[0138] "Manual extraction of matching tables" is the final output of this embodiment, referring to the types of spreadsheets for which the system recommends that manual data extraction and verification must be arranged or given priority in the standardized processing flow of the table. The decision logic is a decision tree containing sub-steps S131-S133.

[0139] This step is the comprehensive evaluation and decision generation layer. It no longer views "number of extractions" or "number of anomalies" in isolation, but combines the two through a set of business rules, introducing concepts such as "daily average frequency" and "total threshold" to make a balanced decision that considers efficiency, data volume, and risk control. Its significance lies in simulating the decision-making process of an experienced manager, achieving automated and intelligent classification of massive amounts of table types through quantitative rules, ensuring that resources (human effort) are accurately allocated to where they are most needed.

[0140] It should be noted that the number of times the extraction results are abnormal refers to the number of times the extraction results are inconsistent with the actual data in the table.

[0141] It is understood that the determination of whether a spreadsheet of this type belongs to a manually extracted matching spreadsheet is based on the historical number of extractions and the number of times the extraction results are abnormal, specifically including:

[0142] S131 determines the average daily number of extractions of the spreadsheet of the type based on the historical extraction count, and determines whether the average daily number of extractions of the spreadsheet of the type is less than a preset threshold. If yes, proceed to the next step; otherwise, the spreadsheet of the type is extracted frequently, and manual extraction is too busy. Therefore, it is determined that the spreadsheet of the type does not belong to the manually extracted matching table.

[0143] This step reflects the principle of "efficiency first." If a type of table needs to be processed many times a day (i.e., high-frequency business), even if it has a certain error rate, relying entirely on manual extraction is impractical and inefficient. In this case, a better strategy is to strengthen automated processes (such as optimizing models and adding post-processing rules) and supplement them with manual spot checks on key points, rather than categorizing it as "manual extraction and matching of tables," which would cause process blockage. Therefore, for high-frequency tables, the system tends to trust and optimize automation rather than reverting to full manual processing.

[0144] S132 determines whether the historical extraction count of the spreadsheet of the type is greater than the preset extraction count threshold. If yes, proceed to the next step. If no, it is determined that the spreadsheet of the type does not belong to the manually extracted matching table because the amount of data is too small.

[0145] This step is a "data significance" test. If a certain type of table has only been processed a handful of times in history (too little data), any statistical indicators calculated based on this (such as the error rate) may lack statistical significance due to insufficient sample size, easily leading to misjudgments. For example, if extraction only occurs once and fails, the error rate is 100%, but this does not reliably prove that this type of table is difficult to process automatically. Therefore, for tables that are "low-frequency and few in number," the system adopts a conservative strategy, not easily marking them as requiring manual processing, but rather continuing to observe or attempting lower-cost automated methods.

[0146] S133 determines the extraction deviation probability of the spreadsheet of the type by the ratio of the number of times the extraction results of the spreadsheet of the type are abnormal to the number of historical extractions, and determines whether the spreadsheet of the type belongs to the manually extracted matching spreadsheet based on the extraction deviation probability.

[0147] This step is the final decision on "risk cost." After the first two screening steps (non-high-frequency, with a certain amount of data accumulation), this step directly focuses on the failure rate of automatic extraction. A high probability of extraction deviation means that each time this type of table is processed automatically, it faces a high risk of error, and the cost of subsequent error correction (manual review, losses caused by business errors) may have exceeded the cost of direct manual extraction. Therefore, when the deviation probability exceeds the business-acceptable risk threshold, the system explicitly recommends using a manual extraction process for this type of table, which is the most direct risk control measure.

[0148] It should be noted that when the extraction deviation probability is greater than the preset deviation probability threshold, the spreadsheet of this type is determined to be a manually extracted matching spreadsheet.

[0149] It is understood that if the spreadsheet of the aforementioned type is a manually extracted matching spreadsheet, then the extraction process for that type of spreadsheet will be performed manually.

[0150] In one possible specific embodiment:

[0151] Suppose that the financial system of an e-commerce company needs to process various expense detail spreadsheets from different departments. The system analyzes one type of spreadsheet: "Overseas Travel and Entertainment Expense Details (Non-standard Version)".

[0152] Step S1 (Data Statistics):

[0153] In the past 90 days, the system has processed this type of form 45 times (historical extraction count).

[0154] Average number of extractions per day = 45 / 90 = 0.5 times / day.

[0155] Step S2 (Anomaly Statistics):

[0156] Of the 45 processing steps, subsequent audits or manual spot checks revealed that the extraction results were inconsistent with the actual data in the table in 18 instances (e.g., currency was confused, or the item names in merged cells were misread). The number of times the extraction results were abnormal was 18.

[0157] Step S3 (Decision Judgment):

[0158] S131: Determine if the average daily number of extractions (0.5 times / day) is less than the preset threshold (assuming it is 5 times / day). If 0.5 < 5, the condition is met, proceed to S132.

[0159] S132: Determine if the number of historical extractions (45 times) is greater than the preset threshold (assumed to be 10 times). If 45 > 10, the condition is met, proceed to S133.

[0160] S133: Calculate the extraction bias probability = 18 / 45 = 40%.

[0161] Determine if the deviation probability (40%) is greater than the preset threshold (assumed to be 15%). If 40% > 15%, the condition is met.

[0162] Final determination: Therefore, the system determines that the "Overseas Travel and Entertainment Expense Details (Non-standard Version)" belongs to the "Manually Extracted and Matched Table".

[0163] Furthermore, the deviation between the extracted data from the spreadsheet and the spreadsheet of the same type is determined based on the deviation between the keywords of the spreadsheet and the spreadsheet of the same type.

[0164] Specifically, such as Figure 4 As shown, the method for determining the generation and management strategy of the prompt words in the spreadsheet is as follows:

[0165] This system aims to establish an intelligent keyword-based strategy decision-making framework for the automated data processing of logistics supplier quotations. Faced with logistics quotations that are varied in format and complex in terminology, the system analyzes historical processing records to build a keyword-based "risk-experience" knowledge base. This allows for the automatic matching of appropriate processing strategies to newly arriving quotations, optimizing processing efficiency while ensuring the accuracy of key information extraction.

[0166] The system follows the principles of "data-driven, risk-based, and content-matching." First, it extracts the keyword features of new quotations; second, it searches for similar "neighbor" combinations in the historical knowledge base; then, it assesses the processing risk of the current quotation based on the historical performance (probability of deviation) of these neighbor combinations; finally, through a multi-level decision tree, it selects between a "high-protection strategy" and a "high-efficiency strategy" based on the risk assessment results, achieving precise "one-quote-one-policy" solutions.

[0167] S21 Based on the historical extracted data of the spreadsheet of the aforementioned type, spreadsheets with consistent keywords are grouped into the same group, and the number of keywords inconsistent with the extracted data of the spreadsheets in the group is determined according to the deviation between the extracted data of the spreadsheets in the group and those of the spreadsheets.

[0168] Logistics quotation history extraction database: Stores the original files of all historical quotation documents, keyword sets extracted by NLP, automatically extracted result data, corrected data after manual review and anomaly markers, processing timestamps and other metadata in a structured manner.

[0169] Domain-Adaptive NLP Model: A natural language processing model optimized for logistics domain terminology (such as "CIF", "DAP", "disposal", "e-release"), used to accurately extract keywords from quotation texts and tables.

[0170] This step is the cornerstone and prerequisite for building the entire intelligent decision-making system. A high-quality domain NLP model ensures the accuracy of keyword extraction, which is the input source for all subsequent matching, clustering, and risk assessment. A rich and clearly labeled historical database provides reliable "experience" samples for statistical analysis and pattern learning. Without accurate "sensors" (models) and sufficient "memory" (data), the system will be unable to perform effective analysis and decision-making. Its significance lies in providing standardized inputs and a reliable source of knowledge for all subsequent intelligent steps.

[0171] Specific examples:

[0172] During system deployment, over 500,000 price quotations from mainstream logistics providers accumulated over the past two years were integrated. Using 100,000 samples that had been precisely annotated manually (including keyword and extraction result accuracy), a dedicated logistics terminology recognition model was trained. This model can accurately identify keywords for specific fee items such as "destination port terminal fees," "documentation fees," and "excess length surcharges."

[0173] Keyword combination / clustering: Group all price lists with identical keyword sets in the historical database into the same group, and define a unique "content pattern" for each group.

[0174] Keyword inconsistency count: A measure of the difference between the keyword set A of the current new quote and the keyword set B_set of a certain historical combination B. It is calculated as |the number of elements in the symmetric difference between A and B_set|, i.e., the total number of words that belong to A but not to B_set, and words that belong to B_set but not to A.

[0175] This step is crucial for achieving "content addressing" and "experience reuse." The complexity and diversity of logistics quotations are reflected in the combination of their service terms and fee structures. Clustering by precise keyword sets is equivalent to creating an independent file for each specific "business scenario." Calculating the "number of inconsistencies" is a simple and effective similarity metric that quantifies the "content distance" between new orders and various historical files. This ensures that when the system searches for historical references, it does so based on similarity at the business semantic level, rather than simply filenames or supplier names, providing a precise anchor for subsequent risk assessments based on specific scenarios.

[0176] Specific examples:

[0177] A newly received quotation for "FCL sea freight from City A to City B, door-to-door, tax included" has the following keywords: {City A, FCL sea freight, City B, door-to-door, DDP}. The historical database contains combination M: {City C, LCL sea freight, City D, port-to-port, FOB}. The number of inconsistencies is calculated: the new order has five keywords not present in M: "City A", "FCL sea freight", "City B", "door-to-door", and "DDP"; M has five keywords not present in the new order: "City C", "LCL sea freight", "City D", "port-to-port", and "FOB". Therefore, the number of inconsistencies is 10. Simultaneously, combination N: {City A, FCL sea freight, a, door-to-door, DDP} is found, with only "City B" differing from "a", resulting in a number of inconsistencies of 2.

[0178] S22 determines the extraction deviation probability of the spreadsheets in the combination based on the historical extraction data of the spreadsheets in the combination;

[0179] Extraction Bias Probability (P): A risk indicator defined for a specific keyword combination (or the entire quote type). The calculation formula is: P = (Total number of times the extraction results of all historical quotes within this combination were marked as "abnormal") / (Total number of times quotes within this combination were processed). It reflects the historical average failure rate of automated processing of quotes with this characteristic.

[0180] The core of this step is to transform discrete historical processing results into continuous risk quantification indicators. In business practice, certain quotation patterns, due to their complex structure, non-standard fields, or ambiguous terminology, are inherently difficult to process automatically. "Bias probability," through statistical analysis of a large number of historical cases, smooths out random errors and reveals the inherent risk level of these patterns. It provides objective and comparable core judgment criteria for decision trees, enabling the system to distinguish between "random errors" and "systemic difficulties," thereby determining whether additional risk control measures are needed for similar new orders. Its significance lies in establishing a bridge from historical performance to future risk prediction.

[0181] Specific examples (continued):

[0182] Historical combination N (characterized by full container load (FCL) door-to-door DDP shipping departing from City A) was processed 180 times, with 15 of those extractions containing errors (e.g., incorrectly calculating "destination port taxes and fees" under DDP terms as 0, or omitting "sealing fees"). Therefore, the extraction error probability for combination N is P_N = 15 / 180 ≈ 8.33%. Simultaneously, the system calculates the overall error probability for the entire "Ocean Quotation" type as P_type = 5%.

[0183] S23 uses the extraction deviation probability of the spreadsheet of the type, the extraction deviation probability of the spreadsheet in the combination, and the number of inconsistencies with the keywords of the spreadsheet to determine the generation management strategy of the prompt words of the spreadsheet.

[0184] Prompt generation management strategy: The final instruction output by the system determines how to construct the prompt and supporting processing logic to guide the large model in data extraction.

[0185] Preset generation strategy: corresponding to high risk / high security mode, that is, when there are a first number (e.g. 9) of prompt words with the same extraction function, the extraction function with the most prompt words with the same extraction function is used for spreadsheet standardization.

[0186] The second preset generation strategy corresponds to the normal / high-efficiency mode. When there are a second number (e.g., 5) of prompt words with the same extraction function, the extraction function with the most prompt words with the same extraction function is used for spreadsheet standardization.

[0187] Preset deviation probability thresholds (T_type, T_combo): Pre-configured decision thresholds, used to determine the overall risk of the type (T_type, e.g., 8%) and the risk of a single combination (T_combo, e.g., 12%), respectively.

[0188] Preset keyword quantity threshold (K): Used to define the leniency of "similar combinations". For example, K=3 means that new orders are allowed to have a maximum of 3 different keywords from historical combinations.

[0189] This step serves as the decision-making center and action guide for the entire methodology. It transforms risk assessment logic into an executable, automated process through a structured, multi-level decision tree. Its design philosophy is a risk exploration strategy that progresses "from tight to loose, from precise to fuzzy," aiming to achieve hierarchical risk response and optimal allocation of computational resources. The decision tree-guided system first focuses on the most severe global risks, then seeks the most accurate historical experience, and finally conducts prudent integrated reasoning when information is incomplete. This design ensures that low-risk, high-certainty business operations can enjoy efficient processing, while high-risk or uncertain business operations receive sufficient prudence, thus achieving the best balance between accuracy, efficiency, and robustness overall.

[0190] It is understood that if the extraction deviation probability of the spreadsheet of the aforementioned type is greater than a preset deviation probability threshold, then the generation management strategy for the prompt words of the spreadsheet is determined to be the preset generation strategy.

[0191] Additionally, it should be noted that if the extraction deviation probability of the spreadsheet of the aforementioned type is not greater than a preset deviation probability threshold, the following content is also included:

[0192] S231 Determine whether there is a combination that is completely consistent with the keywords of the spreadsheet. If so, determine the generation management strategy of the prompt words of the spreadsheet based on the extraction deviation probability of the combination that is completely consistent with the keywords of the spreadsheet. If not, proceed to the next step.

[0193] Determining whether a completely identical keyword combination exists is the first exact match check in the decision tree. If a completely identical combination exists, it means the new quote is a "twin" of a batch of historical tables in terms of content characteristics. The historical experience accumulated by this combination has the highest reference value and predictive accuracy. Prioritizing the results of exact matches minimizes uncertainty and allows for the most reliable strategy selection. This reflects the system's primary principle of pursuing decision-making accuracy.

[0194] Specific examples (continued from S21 / 22):

[0195] For the newly received quotation "City A to City B...", the system search revealed that there is no historical keyword combination that is also exact: {City A, Full Container Load (FCL), City B, Door-to-Door, DDP}. Therefore, no completely identical combination exists, and the decision-making process proceeds to S232.

[0196] If the extraction deviation probability of a combination that is completely identical to the keywords in the spreadsheet is greater than a preset deviation probability threshold, then the generation management strategy for the prompt words in the spreadsheet is determined to be a preset generation strategy. If the extraction deviation probability of a combination that is completely identical to the keywords in the spreadsheet is not greater than a preset deviation probability threshold, then the generation management strategy for the prompt words in the spreadsheet is determined to be a second preset generation strategy.

[0197] S232 determines whether there are combinations of keywords that do not match the number of different combinations of keywords in the spreadsheet, and whether there are combinations that do not match the number of keywords in the spreadsheet, and the number of such combinations is less than a preset keyword number threshold. If so, proceed to the next step; otherwise, determine that the generation management strategy for the prompt words of the spreadsheet is the preset generation strategy.

[0198] Finding similar combinations (number of keyword inconsistencies < K): This step aims to find similar combinations when no identical combinations can be found. In real-world business scenarios, perfectly matching combinations are rare due to subtle differences in suppliers, destinations, and specific terms. Setting a threshold K to define "similarity" is crucial for striking a balance between matching accuracy and coverage. It allows the system to leverage historical experience with semantically similar matches, avoiding falling into the "most conservative strategy" or "no strategy available" situation due to a lack of precise matches, thus enhancing the system's usability and flexibility.

[0199] Specific examples:

[0200] Assume K=3. The number of inconsistencies between the new single and combination N is 2 (City B vs. City A), 2 < 3. Therefore, combination N is identified as a "similar combination". The number of inconsistencies with combination M is 10, far greater than 3, and therefore it is not considered a similar combination.

[0201] S233 identifies combinations where the number of inconsistencies with the keywords in the spreadsheet is less than a preset keyword quantity threshold as similar combinations, and determines the generation management strategy for the prompt words in the spreadsheet based on the extraction deviation probability of the spreadsheets in the similar combinations.

[0202] Furthermore, based on the extraction deviation probability of the spreadsheets in the similar combinations, a generation and management strategy for the prompt words of the spreadsheets is determined, specifically including:

[0203] S234 Determine whether there is a similar combination among the similar combinations whose extraction deviation probability is greater than a preset deviation probability threshold. If yes, determine that the generation management strategy of the prompt words of the spreadsheet is the preset generation strategy. If no, proceed to the next step.

[0204] Based on the initial risk assessment of similar combinations, steps S233 / S234 perform rapid risk screening after identifying similar combinations. The logic is: if any similar combination itself has a high historical risk (probability of deviation > T_combo), then new orders with similar combinations are also very likely to face the same high risk, therefore a high-protection strategy should be directly activated. This is an efficient heuristic rule that can intercept obviously high-risk cases at the first moment without performing more complex calculations, thus improving the system's response speed.

[0205] The deviation probability of combination N is P_N ≈ 8.33%, and the preset T_combo = 12%. Since 8.33% < 12%, the condition "there are similar combinations with a deviation probability greater than the threshold" is not met. Therefore, the decision-making process continues to the more refined S235 step.

[0206] S235 uses the historical extraction count of spreadsheets in different similar combinations as a basis, performs normalization processing to obtain the weight coefficients of different similar combinations, and determines the correction deviation probability based on the sum of the products of the weight coefficients of different similar combinations and the extraction deviation probability. It then determines whether the correction deviation probability is greater than a preset correction probability threshold. If so, the generation management strategy for the prompt words of the spreadsheet is determined to be the preset generation strategy; otherwise, the generation management strategy for the prompt words of the spreadsheet is determined to be the second preset generation strategy.

[0207] Weighting coefficient: The weight is calculated by normalizing the number of times each similar combination has been processed in its history. The more times the combination has been processed, the more stable its statistical results are, and the higher its weight.

[0208] Corrected deviation probability: A weighted average risk value calculated by comprehensively considering the risk levels and weights of all similar combinations. The formula is: Corrected deviation probability = Σ(Probability of deviation for each similar combination * Weight coefficient of that combination).

[0209] This step is the most refined and robust decision-making stage in the decision tree. When the risk performance of multiple similar combinations is inconsistent, simple "if it exists, determine" or "take the maximum value" approaches may be inaccurate. S235 introduces a weighted comprehensive evaluation, the significance of which lies in its reliance on historical experience with a larger dataset. A combination processed 1000 times with a deviation rate of 10% has a far more reliable risk signal than a combination processed only 10 times with a deviation rate of 20%. Through weighted calculation, the system derives a more stable and reliable comprehensive risk prediction value. This demonstrates the system's advanced ability to make rational decisions and quantify risk in uncertain environments, avoiding being misled by a small number of outlier data points.

[0210] Specific examples:

[0211] Suppose that in addition to combination N (processed 180 times, P=8.33%), another similar combination Q (with slightly different keywords, processed 20 times, P=15%) is found. Calculate the weights: Total number of processes = 180 + 20 = 200. Weight of combination N = 180 / 200 = 0.9; Weight of combination Q = 20 / 200 = 0.1. Calculate the corrected bias probability = (8.33% * 0.9) + (15% * 0.1) = 7.5% + 1.5% = 9.0%.

[0212] Judgment: Assume the preset correction probability threshold is 10%. Since 9.0% < 10%, the correction deviation probability is not greater than the threshold. Final decision: According to rule S235, the "second preset generation strategy" is adopted for this new quotation.

[0213] Furthermore, the preset generation strategy is that when there are a first number of prompt words with the same extraction function, the extraction function with the most prompt words with the same extraction function is used for spreadsheet standardization.

[0214] Furthermore, the second preset generation strategy is that when there are a second number of prompt words with the same extraction function, the extraction function with the most prompt words with the same extraction function is used for spreadsheet standardization.

[0215] Furthermore, the generation of prompt words for the large language model based on the aforementioned generation management strategy specifically includes:

[0216] When the generation management strategy is a preset generation strategy, the first number of prompt words are used each time to generate prompt words for the large language model.

[0217] When the generation management strategy is the second preset generation strategy, the second number of prompt words are used each time to generate prompt words for the large language model.

[0218] It should be further explained that if the number of prompt words generated in the spreadsheet exceeds the number of prompt words generated in a single instance corresponding to the preset multiple generation management strategy, a stop control strategy will be determined to avoid the technical problem of excessive extraction time caused by the inability to meet the requirements of the generation management strategy for a long time.

[0219] It should be noted that the value of the preset multiple is between 2 and 3.

[0220] Specifically, the method for determining the termination control strategy for the extracted prompts in the spreadsheet is as follows:

[0221] The core objective of this embodiment is to address the challenge of dynamically deciding when to stop generating new suggestions during spreadsheet automation, based on the distribution of extraction functions corresponding to the real-time generated suggestions. This avoids infinite loops and inefficiency caused by the inability to achieve the required function consistency (either a first quantity of 9 or a second quantity of 5) as mandated by management strategies. The core logic is as follows: suggestions with the same extraction function are grouped together. By monitoring the size of each extraction function group (i.e., the number of supported suggestions) and its distribution concentration in real time, the system determines whether it is possible to achieve the management strategy requirements within a reasonable cost. The system sets three total quantity thresholds (high, medium, and low) based on distribution characteristics. When a function is detected to have an absolute advantage, more suggestions are allowed to be generated to quickly reach consensus. When the distribution is extremely dispersed, the number of suggestions generated is strictly controlled to avoid waste. When multiple competing functions exist, a moderate exploration space is provided. Finally, when the number of suggestions generated reaches the preset threshold, the process is forcibly stopped to ensure timeliness.

[0222] S31 uses the extracted function group data to determine the number of extracted function groups;

[0223] "Data on extraction functions corresponding to generated prompt words" refers to all prompt words that the system has generated for the current table task and the specific extraction function identifiers they are bound to; "Extraction function groups" are sets of all prompt words bound to the same extraction function, with each group corresponding to a unique extraction function; "Number of extraction function groups" refers to the total number of different extraction functions involved in the currently generated prompt words.

[0224] This step forms the real-time state awareness and structured foundation for dynamic decision-making. By grouping the generated prompts according to their extraction functions, the system can clearly grasp the current distribution of "voting": how many different "candidate solutions" (functions) there are, and how many "support votes" (prompts) each solution has received. This is the direct basis for assessing whether the system is close to achieving the management strategy requirements (e.g., requiring 9 votes for the same function). Without this real-time grouping statistics, the system cannot determine the progress of consensus formation.

[0225] Specific example: To process a logistics quotation, the system has generated 15 prompt words. After grouping by extraction function, it forms 4 groups: the parse_table_v2 function group (6 prompt words), the regex_price_extract function group (4 prompt words), the ocr_parser function group (3 prompt words), and the legacy_adapter function group (2 prompt words). The total number of extraction function groups is 4.

[0226] S32 determines the proportion of prompt words in the prompt words of different extraction function groups in the prompt words of the spreadsheet, and uses it as the proportion of prompt words in the extraction function groups;

[0227] "Prompt Quantity Ratio" refers to the percentage of prompt words in a specific extraction function group out of the total number of prompt words generated so far. The calculation formula is: (Number of prompt words in a function group) / (Total number of generated prompt words) × 100%.

[0228] This step aims to quantify the relative support and influence of each candidate function in the current exploration process. The function with the highest proportion is the "leader" most likely to reach consensus. Analyzing the proportion distribution of all functions can determine the degree of concentration in consensus formation: whether it is highly concentrated on a single leading function or relatively evenly distributed among multiple functions. This provides crucial information for predicting whether and how many additional generation attempts are needed to reach the target number (9 or 5).

[0229] Specific example: The total number of generated prompt words is 15. The proportions of each function group are calculated as follows: parse_table_v2 group proportion = 6 / 15 = 40%; regex_price_extract group proportion = 4 / 15 ≈ 26.7%; ocr_parser group proportion = 3 / 15 = 20%; legacy_adapter group proportion = 2 / 15 ≈ 13.3%.

[0230] S33 determines the termination control strategy for extracting prompts from the spreadsheet based on the number of extraction function groups and the proportion of prompt words in different extraction function groups.

[0231] The "Generation Stop Threshold" is a system-preset upper limit on the total number of prompt words generated for different distribution scenarios. It is divided into "Target Quantity Threshold" (N1, highest), "Second Target Quantity Threshold" (N2, lowest), and "Third Target Quantity Threshold" (N3, medium), satisfying N1 > N3 > N2. The "Stop Control Strategy" determines which threshold to use as the upper limit for this generation task. When the cumulative number of generated words reaches this threshold, generation must stop regardless of whether the management strategy requirements have been met.

[0232] This step is the core control mechanism to prevent unlimited generation and ensure efficiency. By analyzing real-time distribution characteristics, the system dynamically selects the most appropriate cost ceiling. Its significance lies in achieving scenario-based adaptation of exploration costs: allocating a higher budget when the chances are high, and strictly controlling losses when the chances are slim. This ensures that the system does not endlessly consume resources on a single task, thereby maintaining the throughput and response speed of the overall processing flow.

[0233] It is understood that, based on the number of extraction function groups and the proportion of prompt words in different extraction function groups, the termination control strategy for the extraction prompt words of the spreadsheet is determined, specifically including:

[0234] S331 determines whether there is an extraction function group whose prompt word ratio is greater than a preset number ratio threshold by using the prompt word ratio of different extraction function groups. If so, the stop control strategy for the extraction prompt words of the spreadsheet is to control the total number of prompt words to the target number threshold. If not, proceed to the next step.

[0235] This step aims to identify scenarios where success is "close" or where there is a clear advantage. When a function receives more than half of the "support," it indicates that it is likely the final solution, with a high probability of reaching the target number (9 or 5). Setting a higher upper limit (N1) at this point aims to provide sufficient "ammunition" to ensure that this advantageous function can eventually accumulate the required number of votes, reflecting the principle of "going with the flow to ensure success."

[0236] Specific example: The preset quantity ratio threshold is 50%. The current leading function parse_table_v2 has a ratio of 40%, which does not exceed 50%. The condition is not met, and the decision process proceeds to S332.

[0237] S332 determines whether the proportion of prompt words in different extraction function groups is less than a preset proportion threshold. If yes, it determines that the stop control strategy for the extraction prompt words of the spreadsheet is to control the total number of prompt words to the second target number threshold. If no, it proceeds to the next step.

[0238] It should be noted that the target quantity threshold is greater than the second target quantity threshold.

[0239] This step aims to identify high-risk scenarios characterized by "extreme dispersion and slow progress." If the support rates for all functions are very low (e.g., all below 15%), it indicates that the generation process is highly dispersed and lacks effective focus. Following this trend, it will be difficult to accumulate 9 or 5 votes for any function within an acceptable cost. In this case, the most conservative strategy should be adopted immediately, setting a minimum upper limit (N²), strictly controlling further investment, and preparing to switch to alternative solutions as early as possible, reflecting the principle of "timely loss mitigation and risk control."

[0240] Specific example: The preset low percentage threshold is 15%. Inspection reveals that the legacy_adapter group has a percentage of 13.3% (<15%), but the other three groups all have percentages greater than 15%. The condition is not met, and the decision process proceeds to S333.

[0241] S333 takes the extraction function group whose proportion of prompt words is not less than the preset proportion threshold as the suspected matching group, and determines whether the number of the suspected matching group is greater than the preset suspected matching group threshold. If so, then there may be a large number of functions that meet the requirements of the generation management strategy. Therefore, the stop control strategy for the prompt words extracted by the spreadsheet is to control the total number of prompt words to the third target number threshold. If not, proceed to the next step.

[0242] This step aims to handle the common competitive scenario of "multiple strong contenders requiring further competition." A "potential matching group" represents functions with a substantial support base and the potential to reach consensus. When there are many such competitors (more than two), it indicates that more generation attempts are needed to allow them to "determine a winner." Setting a moderate upper limit (N3) at this point provides reasonable additional exploration space, aiming to select a winner from among multiple promising candidates, reflecting the principle of "full competition, selecting the best."

[0243] Specific example: There are 3 suspected matching groups (proportion ≥ 15%): parse_table_v2 (40%), regex_price_extract (26.7%), and ocr_parser (20%). The preset threshold for the number of suspected matching groups is 2. 3 > 2, so the condition is met. Therefore, the termination control strategy is determined to be: the total number of prompt words is controlled at the third target threshold (N3). Assuming N3 = 35, the system will generate a maximum of 20 more (35-15) prompt words.

[0244] S334 determines the distribution dispersion coefficient of the prompt words based on the number of extraction function groups and the proportion of prompt words in different extraction function groups, and determines the termination control strategy for the extraction prompt words of the spreadsheet based on the distribution dispersion coefficient.

[0245] It is understood that the distribution dispersion coefficient of the prompt words is determined by the number of extraction function groups and the proportion of prompt words in different extraction function groups. The more extraction function groups there are and the smaller the proportion of prompt words in each extraction function group, the larger the distribution dispersion coefficient of the prompt words.

[0246] Calculate the dispersion coefficient of the prompt words among the function groups. If it is greater than the preset threshold, use N2; otherwise, use N3.

[0247] This step is a refined fallback decision. When the number of suspected matching groups is at a critical value, by calculating the distribution dispersion coefficient, a continuous statistic, we can more accurately determine whether the distribution tends to be dispersed (large coefficient, use N2 stop loss) or relatively concentrated (small coefficient, use N3 to continue exploring).

[0248] Specifically, the termination control strategy for determining the extracted prompts from the spreadsheet based on the distribution dispersion coefficient includes:

[0249] If the distribution dispersion coefficient is greater than a preset dispersion coefficient threshold, then the termination control strategy for the extracted prompt words in the spreadsheet is determined to be that the total number of prompt words is controlled at a second target number threshold; otherwise, the termination control strategy for the extracted prompt words in the spreadsheet is determined to be that the total number of prompt words is controlled at a third target number threshold.

[0250] It should be noted that the formula for calculating the coefficient of variation is as follows:

[0251] G: The number of function groups extracted (G ≥ 1);

[0252] P_i: The proportion of prompt words in the i-th extraction function group (i = 1, 2, ..., G);

[0253] It satisfies: ∑_{i=1}^{G} P_i = 1, and 0 ≤ P_i ≤ 1.

[0254] It should be noted that the third target quantity threshold is greater than the second target quantity threshold (e.g., 30) and less than the target quantity threshold (e.g., 40).

[0255] Furthermore, the matching status of the extraction function with the generation management strategy is determined based on whether there is an extraction function that satisfies the generation management strategy when the threshold of the total number control corresponding to the termination control strategy is reached.

[0256] Specifically, the method for determining the update method of the manually extracted matching table is as follows:

[0257] The core objective of this embodiment is to establish a data-driven intelligent decision-making system to dynamically determine which spreadsheet types should be included in the "manually extracted matching spreadsheets" list based on the actual performance (success rate and resource consumption) of automated processing of various spreadsheet types, thereby optimizing the allocation of human and automated processing resources. Its core logic is as follows: by continuously collecting the processing results (matching success or failure) and the number of prompt words consumed in successful processing for each type of spreadsheet, the system quantitatively evaluates the reliability (matching deviation rate) and processing cost (average number of generated spreadsheets) of automated processing for each type of spreadsheet. Based on this performance data, the system executes a hierarchical decision-making process: when a widespread performance problem is detected, a proactive strategy (low failure rate threshold) is adopted to quickly expand the scope of manual support to control system risk; when the problem exists locally, a refined cost-benefit analysis is conducted based on processing costs, using a lower threshold to include spreadsheet types with "high cost and insufficient reliability" in manual processing, while using a higher threshold for other situations for prudent decision-making, thereby ensuring that human resources are accurately invested in the weakest link of automated processing efficiency.

[0258] S41 uses the matching status of the spreadsheet to determine the spreadsheets of the type whose matching status does not meet the requirements, and identifies them as matching deviation spreadsheets.

[0259] "Matching status" refers to whether the output of the spreadsheet meets the expected standards after the complete automated extraction process. Specifically, it is determined by whether there is an extraction function that meets the requirements of the generation management strategy (first quantity 9 or second quantity 5) when the total number threshold set by the stop control strategy is reached. "Matching status does not meet the requirements" means that no extraction function that meets the conditions was found after the quantity limit was reached, and the processing failed. "Matching deviation spreadsheet" refers to these instances of spreadsheets that failed to process.

[0260] This step is the foundational data collection and problem quantification stage for performance evaluation. The criteria for determining the matching status directly link process control (suspension strategy) and result verification (management strategy), ensuring the objectivity and consistency of the evaluation criteria. By statistically analyzing the number of failure cases under each type, the failure rate of automated processing for that type can be directly calculated. This is the most crucial indicator for measuring the reliability of automated processing, providing direct "quality defect" evidence for subsequent decisions on whether to switch to manual processing.

[0261] Specific example: In the previous statistical period, the system processed 180 "Supplier Quotation" type spreadsheets. According to the matching judgment rules, 24 of these spreadsheets failed to find a matching function with at least 5 prompts pointing to the same function (using the second preset strategy) after the number of generated prompt words reached the upper limit, and were therefore marked as failures. These 24 spreadsheets are the matching deviation spreadsheets of this type.

[0262] S42 determines the type of spreadsheet based on the generated prompt words of the spreadsheet, and the number of prompt words generated when the management strategy is required is reached;

[0263] "When the generation management policy requirements are met" specifically refers to those table instances that are successfully matched; "Number of prompt words generated" refers to the total number of prompt words consumed by each successful instance when generation stops, and its upper limit is constrained by the stop control policy (N1 / N2 / N3); "Average number of prompt words generated" is the arithmetic mean of the number of prompt words generated by all successful instances of the same type.

[0264] This step aims to quantify the efficiency cost or resource consumption of automated processing. The average number of forms generated reflects the "cost" of successfully processing a particular type of form. Even if successful, high costs may indicate poor economic viability of automation. This provides a key basis for assessing whether continued automation is worthwhile from an "input-output ratio" perspective. Combined with failure rate metrics, it can more comprehensively identify those form types that are "difficult to succeed and costly to succeed."

[0265] For example, in the 156 successfully processed "supplier quotation" cases mentioned above, statistics showed that an average of 32 prompts were needed per form before consensus was reached. This indicates that the automation resource cost for processing such forms is relatively high.

[0266] S43 determines the update method for the manually extracted matching table based on the number of matching deviation tables in the spreadsheets of the aforementioned type and the number of prompt words generated in different spreadsheets.

[0267] "Manual extraction and matching table update method" refers to the decision rules output by the system, specifically, selecting a failure rate threshold: when the failure rate of a certain type of table exceeds this threshold, it is included in the manual processing list. The system defines two thresholds: "preset table percentage threshold" (Th_low, lower, such as 10%) and "second preset table percentage threshold" (Th_high, higher, such as 15%).

[0268] This step serves as the intelligent hub for comprehensive assessment and strategy formulation. It avoids "one-size-fits-all" decision-making, achieving dynamic responses to both global system risks and local problem characteristics through a multi-level decision tree. Its core significance lies in balancing the agility of risk control with the precision of resource optimization: rapid response to general risks (using low thresholds), and prudent decision-making after in-depth cost-benefit analysis of local problems (flexibly selecting high or low thresholds), thereby guiding human resources precisely to areas where automation efficiency is lacking.

[0269] Furthermore, if there are matching deviation spreadsheets in different types of spreadsheets, then the update method for manually extracting matching spreadsheets is determined to be that when the proportion of matching deviation spreadsheets in the spreadsheets of that type is greater than a preset spreadsheet proportion threshold (e.g., 10%), then the spreadsheets of that type are determined to be manually extracted matching spreadsheets.

[0270] Additionally, it is understandable that if there are unevenly matched spreadsheets of different types, including the following:

[0271] S431 Based on the proportion of matching deviation spreadsheets in different types of spreadsheets, determine whether there is a type of spreadsheet whose proportion of matching deviation spreadsheets is greater than a preset table proportion value (less than a preset table proportion threshold). If yes, proceed to the next step; otherwise, determine that the update method of the manually extracted matching table is that when the proportion of matching deviation spreadsheets in the spreadsheet of the type is greater than a second preset table proportion threshold (greater than the preset table proportion threshold), the spreadsheet of the type is determined to be a manually extracted matching table.

[0272] This step serves as a risk warning and key target identification. A "table percentage preset value" (e.g., 8%) is set, which is below the final decision threshold but above the normal level. This is used to identify early-stage failures that show a negative trend but have not yet reached a severe level. This helps the system focus resources on conducting a more in-depth analysis of these potential risk points, reflecting a proactive management approach.

[0273] For example: The failure rate for "Supplier Quotation" is approximately 13.3% (24 / 180). Assume there is also a "Project Budget" type with a failure rate of 6%. Let the preset percentage for each spreadsheet be 8%. Since "Supplier Quotation" has a higher percentage (13.3% > 8%), this indicates a mismatch, and we proceed to the next step.

[0274] S432 determines whether the number of types of electronic tables whose proportion of the number of matching deviation electronic tables is greater than the preset table proportion value is greater than the preset type number threshold. If yes, it is determined that the update method of the manually extracted matching table is that when the proportion of the number of matching deviation electronic tables in the electronic tables of the type is greater than the preset table proportion threshold, it is determined that the electronic tables of the type belong to the manually extracted matching tables. If not, proceed to the next step.

[0275] This step assesses the local concentration and breadth of the problem. If multiple types (more than the threshold, e.g., two) simultaneously exhibit moderately high failure rates, this may indicate a cluster of problems affecting a specific business area. Although not global, the impact is still relatively broad, therefore a more proactive approach should be adopted, namely, updating the manual checklist with a lower failure rate threshold (Th_low) to control this localized risk.

[0276] Specific example: Suppose that only the "Supplier Quotation" type satisfies condition S431. The preset threshold for the number of types is 2. Since 1 is not greater than 2, the condition is not met, and the decision process proceeds to the final S433.

[0277] S433 determines the filter table type in the specified type of spreadsheet based on the proportion of matching deviation spreadsheets in different types of spreadsheets and the average number of prompt words generated in different spreadsheets. Based on the filter table type, it determines the update method for the manually extracted matching table.

[0278] It should be noted that the filter table type is the type of electronic form whose average number of generated prompt words for different electronic forms is greater than the preset generation threshold, and whose proportion of matching deviation electronic forms is within the preset range.

[0279] "Filtering table types" here specifically refers to "delayed generation table types," which are defined as table types where the average number of generated prompts exceeds a preset generation threshold (e.g., 30), and the matching deviation rate (failure rate) is within a preset range (e.g., higher than 5%). Automating this type of table is costly and has a certain degree of unreliability, making it a key consideration in cost-benefit trade-offs.

[0280] This step is the final decision in the cost-benefit analysis. It focuses on those "neither good nor bad" types—those that automation can handle but are costly and not entirely reliable. If there are many of these "high-cost, medium-risk" types (more than the preset number of table types), the overall economics of the automation process are challenged, and the criteria should be relaxed (using a lower Th_low) to encourage more types to be handled manually to save overall resources. If there are few of these types, the problem is isolated or the cost is acceptable, and the criteria should be tightened (using a higher Th_high), with manual intervention only for the most severe failures to maximize the coverage of automation.

[0281] Assume the "Supplier Quotation" type has a failure rate of 13.3% and an average of 32 quotations generated.

[0282] Assume the preset generation threshold is 20, and the preset failure rate is greater than 5%.

[0283] This type satisfies both conditions (32>20, and 13.3%>5%), and is therefore marked as a "delayed table generation type".

[0284] Count the number of table types marked as "delayed generation table type" among all table types. Assume the total number is only 2, and the default value for the number of table types is 3.

[0285] Since 2 is no greater than 3, the system ultimately determined the update method to be: adopting the stricter "second preset table percentage threshold (Th_high=15%)". For the "supplier quotation" type, its failure rate of 13.3% does not exceed 15%, so it will not be included in the manual extraction and matching table for the time being, but it will continue to be a key focus due to its high cost.

[0286] It should be noted that if the number of the selected table types is greater than the preset value for the number of table types, the update method for manually extracting matching tables is determined to be that when the proportion of matching deviation tables in the type of spreadsheet is greater than the preset table proportion threshold, the type of spreadsheet is determined to be a manually extracted matching table. Conversely, if the number of the selected table types is not greater than the preset value for the number of table types, the update method for manually extracting matching tables is determined to be that when the proportion of matching deviation tables in the type of spreadsheet is greater than the second preset table proportion threshold (greater than the preset table proportion threshold), the type of spreadsheet is determined to be a manually extracted matching table.

[0287] Example 3

[0288] Thirdly, the present invention provides a computer system comprising: a memory and a processor connected in communication, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the above-described spreadsheet standardization processing method based on a large language model when running the computer program.

[0289] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0290] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0291] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A spreadsheet standardization processing system based on a large language model, characterized in that, Specifically, it includes: The file acquisition module is responsible for obtaining the file path and basic information of the spreadsheet. The structure parsing module extracts a list of names of all worksheets in the spreadsheet to obtain the type of the spreadsheet. Based on the historical extraction deviation data of the type of the spreadsheet, when it is determined that the spreadsheet does not belong to the manually extracted matching table, the name list is input into the big language model workflow. Combined with domain keywords, semantic relevance analysis is performed on the worksheets to intelligently filter out the target worksheet set containing valid data. The slicing module is responsible for partitioning the worksheet content to obtain multiple data slices. The header and sample data of each data slice are input into the large language model workflow to intelligently identify the dimension type and structural features of the data. The execution module is responsible for determining the generation and management strategy of the prompt words for the spreadsheet based on the deviation between the extracted data and the spreadsheet of the specified type, as well as the historical extracted data of the spreadsheet of the specified type. Based on the generation and management strategy and the prompt words, it calls the large language model to dynamically generate targeted data processing function code, executes the generated function code through the function execution engine, performs standardization transformation on the data slices, and merges and exports all successfully processed data into a standardized spreadsheet. Based on the similarity of extraction functions corresponding to different prompt words, the same extraction functions are grouped into the same extraction function group. Based on the extraction function group data and the prompt words in different extraction function groups, the termination control strategy for the extraction prompt words of the spreadsheet is determined. After determining that the spreadsheet of the aforementioned type has reached the stop control strategy, the matching status of the extraction function with the generation management strategy is determined. Based on the matching status of different spreadsheets and the generation data of prompt words, the update method of the manually extracted matching spreadsheet is determined. The method for determining the management strategy for generating prompts in the spreadsheet is as follows: Based on the historical extracted data of the spreadsheets of the aforementioned type, spreadsheets with consistent keywords are grouped into the same group. The number of keywords that are inconsistent with those of the spreadsheets in the group is determined according to the deviation between the extracted data of the spreadsheets in the group and those of the spreadsheets in the group. Based on the historical extraction data of the spreadsheets in the combination, determine the extraction deviation probability of the spreadsheets in the combination; By utilizing the extraction deviation probability of the spreadsheet of the aforementioned type, the extraction deviation probability of the spreadsheets in the combination, and the number of inconsistencies with the keywords of the spreadsheet, a generation management strategy for the prompt words of the spreadsheet is determined.

2. The spreadsheet standardization processing system based on a large language model as described in claim 1, characterized in that, The keywords in this field include logistics, pricing, and transportation.

3. The spreadsheet standardization processing system based on a large language model as described in claim 1, characterized in that, The worksheet content includes cell data, merged cell information, and the original layout structure.

4. The spreadsheet standardization processing system based on a large language model as described in claim 1, characterized in that, Partitioning the worksheet content includes: Automatic layout type detection determines whether a layout is horizontal or vertical by statistically analyzing the distribution characteristics of key identifiers in rows and columns. Intelligent data boundary identification combines empty row / column detection, keyword matching, and header feature recognition to determine the boundaries of each data block; Multi-pattern matching algorithm: It uses matching algorithms including exact matching, prefix matching, and inclusion matching to identify the location of data identifiers, and supports name variations and additional information.

5. The spreadsheet standardization processing system based on a large language model as described in claim 1, characterized in that, After obtaining the data processing function code, the generated data processing function code is saved to the function library to support subsequent rapid reuse and version management.

6. The spreadsheet standardization processing system based on a large language model as described in claim 1, characterized in that, The type of the spreadsheet is determined based on the parsing results of the spreadsheet's header.

7. The spreadsheet standardization processing system based on a large language model as described in claim 1, characterized in that, Determining that the spreadsheet is not a manually extracted matching spreadsheet includes: Based on the historical extraction data of the spreadsheet of the aforementioned type, determine the historical extraction count and the average daily extraction count of the spreadsheet of the aforementioned type; Based on the historical extraction deviation data, determine the number of times the extraction results of the spreadsheet of this type are abnormal; Based on the historical number of times the spreadsheet of this type was extracted and the number of times the extraction results were abnormal, it was determined whether the spreadsheet belonged to the manually extracted matching spreadsheet.

8. The spreadsheet standardization processing system based on a large language model as described in claim 7, characterized in that, The number of times the extraction results are abnormal refers to the number of times the extraction results are inconsistent with the actual data in the table.

9. The spreadsheet standardization processing system based on a large language model as described in claim 8, characterized in that, The method for determining the update method of the manually extracted matching table is as follows: Based on the matching status of the spreadsheets, identify the spreadsheets of the type that do not meet the matching requirements and designate them as matching deviation spreadsheets. Based on the generated prompts data of the spreadsheet, determine the type of spreadsheet and the number of prompts generated when the management strategy requirements are met; The update method for the manually extracted matching table is determined based on the number of matching deviation tables in the aforementioned type of spreadsheet and the number of prompt words generated in different spreadsheets.

Images