Spreadsheet automatic operation and reasoning method and system based on large language model

Abstract

The invention provides an automatic spreadsheet operation and reasoning method and system based on a large language model, and the method comprises the steps: S1, uploading a spreadsheet file, and extracting related information; s2, the table state, the table description and the system prompt of the information provider module are provided for the information provider module; s3, generating an SQL query statement, executing the SQL query statement, and generating a sub-table corresponding to the spreadsheet; s4, extracting forward data from the sub-table, and providing the forward data to the planner module in combination with a table state, table description and a system prompt of the planner module; s5, generating a Python code through a planner module, and executing the Python code in an interpreter to obtain an execution result; s6, judging and skipping according to an execution result until the code is successfully executed; and S7, repeating the steps S1 to S6 until the task is completed. The method can effectively improve the capability and efficiency of automatically solving spreadsheet operation and reasoning tasks.

Description

Technical Field

[0001] The present invention relates to an automatic operation method of an electronic form, in particular to an automatic operation and reasoning method of an electronic form based on a large language model, and further to a system adopting the automatic operation and reasoning method of an electronic form based on a large language model. Background Art

[0002] Spreadsheet data is crucial in fields such as scientific research, finance, and education. People usually use systems such as Excel and WPS to complete spreadsheet tasks, including numerical calculations, data analysis, and visualization. However, processing these tasks usually involves a lot of repetitive labor and consultation, which takes a lot of time.

[0003] With the popularity of large language models, it has become a trend to use them to build intelligent agents to help humans solve problems automatically. The core process is to construct prompt words, use large language models to plan tasks and call tools to complete complex tasks. At present, intelligent agents based on large language models have been widely used in various fields of production and life, such as intelligent report generation, intelligent customer service, and scientific research assistants. For example, frameworks such as Sheet Copilot and Instruct GPT are currently well-known spreadsheet automation operation frameworks based on large language model agents, which can allow large language models to generate custom API interfaces and Office Scripts code (Excel script conversion code) to manipulate spreadsheets. However, although this existing tool can simplify task requirements in the real world, it cannot cope with reasoning challenges such as fuzzy expressions and multi-step reasoning, and it is difficult to solve more complex spreadsheet tasks, nor can it solve table reasoning tasks. Therefore, it is necessary to provide new technical solutions and solve the above-mentioned technical problems by combining technical features. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an electronic spreadsheet automatic operation and reasoning method based on a large language model, which aims to realize the collaborative work between the information provider module, the planner module and the retriever module based on the large language model and through the steps of table information extraction, sub-table generation, data extraction, code generation and execution, execution status judgment and loop, so as to cope with more complex electronic spreadsheet operation and reasoning tasks, and improve the ability and efficiency of automatically solving electronic spreadsheet operation and reasoning tasks. On this basis, a system using the electronic spreadsheet automatic operation and reasoning method based on a large language model is further provided.

[0005] In this regard, the present invention provides an electronic spreadsheet automated operation and reasoning method based on a large language model, comprising the following steps:

[0006] Step S1, uploading an electronic form file and a user instruction, and extracting relevant information from the electronic form file, the relevant information including form status, form description, system prompts of the information provider module, and system prompts of the planner module;

[0007] Step S2, providing the table state, table description and system prompt of the information provider module extracted in step S1 to the information provider module implemented based on the large language model in the intelligent agent;

[0008] Step S3, generating an SQL query statement based on the information provided in step S2, and executing it in the database to generate a subtable corresponding to the uploaded electronic form;

[0009] Step S4, extracting the first k rows of data from the subtable generated in step S3, combining the table state, table description and system prompt of the planner module extracted in step S1, and providing them to the planner module implemented based on the large language model in the agent;

[0010] Step S5, generating Python code through the planner module according to the information provided in step S4, and executing it in the interpreter to obtain the code execution result;

[0011] Step S6, judging according to the code execution result obtained in step S5, if the execution is successful, the code is executed on the target electronic form, and the electronic form state is updated to proceed to the next step of planning; if the execution fails, the retriever module in the agent retrieves similar code fragments to assist the planner module in regenerating the code until the code execution is successful;

[0012] Step S7, repeating steps S1 to S6 until the task is completed, and saving the electronic spreadsheet after the operation.

[0013] A further improvement of the present invention is that the intelligent agent includes an information provider module, a planner module and a retriever module, wherein the information provider module and the planner module are implemented based on a large language model; the information provider module is used to parse task requirements and generate SQL query statements related to the task so as to extract sub-table information and send it to the planner module in combination with some relevant information of the spreadsheet file; the planner module is used to translate the task semantics into Python code so as to implement code execution and operation; the retriever module is used to retrieve sample code for spreadsheet file operations from a code library to assist the planner module in generating Python code.

[0014] A further improvement of the present invention is that the implementation process of extracting relevant information from the electronic spreadsheet file in step S1 includes the following sub-steps:

[0015] Step S101, the user selects and uploads an electronic spreadsheet file through the interface. After receiving the electronic spreadsheet file, first verify whether the electronic spreadsheet file format is a pre-set supported electronic spreadsheet format. If the format is incorrect, prompt the user to re-upload it. Jump to step S102 until the electronic spreadsheet file format verification passes;

[0016] Step S102, parsing the uploaded spreadsheet file, using the pandas data processing and analysis library of the scripting language Python to read the data in the spreadsheet file and converting it into a data frame DataFrame format;

[0017] Step S103, according to the obtained data frame, extract and record the corresponding table status, table description, system prompt of the information provider module and system prompt of the planner module, the table status is used to record the number of rows, columns, names of each column and data type of each table data in the uploaded electronic spreadsheet file, and the table status is extracted through the object attribute acquisition function of the scripting language Python; the table description is used to record the content or theme summary of the uploaded electronic spreadsheet file, and the table description is extracted through the uploaded natural language; the system prompt of the information provider and the system prompt of the planner module are used to prompt the identification information of the information provider and the planner module respectively, and are extracted through the provided hard-coded form.

[0018] A further improvement of the present invention is that in step S3, the specific process of generating an SQL query statement based on the information provided in step S2 includes: storing all table data of the current state of the spreadsheet file in the database, integrating the information provided in step S2 into the command symbol Prompt and inputting it into the large language model, and generating an SQL query statement through the large language model; the generated SQL query statement is executed in the database and a result is obtained, and the result is used as a subtable corresponding to the uploaded spreadsheet.

[0019] A further improvement of the present invention is that the SQL query statement q is used to generate the SQL query statement through the large language model. t =Informer(q t |P I ,I,A t-1 ,s t ), where Informer(q t |*) represents the generated SQL query statement, P I Indicates the system prompt of the information provider module, I indicates the task instruction, A t-1 represents the reasoning trajectory of the planner module, s t Indicates the table status.

[0020] A further improvement of the present invention is that, in step S4, the implementation process of extracting the first k rows of data from the subtable generated in step S3 includes the following sub-steps:

[0021] Step S401, using the pandas data processing and analysis library of the scripting language Python to read the content of the spreadsheet file and convert it into a data frame format;

[0022] Step S402: extract the first k rows of data in the sub-table through the slice operation instruction df[:k] of the pandas data processing and analysis library.

[0023] A further improvement of the present invention is that in step S5, the specific process of the planner module generating Python code includes: integrating the information provided in step S4 into the command symbol Prompt and inputting it into the large language model, and generating Python code through the large language model; the generated Python code is executed in the virtual sandbox environment of the interpreter, and the generated result is binary information, which is used to represent correct or incorrect respectively.

[0024] A further improvement of the present invention is that in step S6, the implementation process of retrieving similar code fragments by the retriever module in the agent to assist the planner module in regenerating the code is as follows:

[0025] Step S601: Execute the wrong code through the Embedding vector mapping model Convert to vector as a query vector;

[0026] Step S602: Input the query vector into the code base Retrieve the top k most similar code snippets

[0027] Step S603: convert the above k code snippets As additional information for the planner module, return to step S5 to have the planner module regenerate Python code.

[0028] A further improvement of the present invention is that the step S601 executes the wrong code Convert to vector The implementation process is as follows:

[0029] Step S6011, use regular expressions to remove codes Comments and extra whitespace in strings;

[0030] Step S6012: Convert the code into a vector by using the encode numerical conversion method of the Embedding vector mapping model.

[0031] The present invention also provides an electronic form automatic operation and reasoning system based on a large language model, which adopts the electronic form automatic operation and reasoning method based on a large language model as described above, and includes:

[0032] An electronic spreadsheet file upload and extraction module, which uploads electronic spreadsheet files and user instructions, and extracts relevant information from the electronic spreadsheet files, the relevant information including form status, form description, system prompts of the information provider module, and system prompts of the planner module;

[0033] An input module, which provides the table status, table description and system prompt of the information provider module extracted from the spreadsheet file upload and extraction module to the information provider module implemented based on the large language model in the intelligent agent;

[0034] The sub-table generation module generates SQL query statements based on the information provided by the input module and executes them in the database to generate sub-tables corresponding to the uploaded electronic forms;

[0035] The data extraction module extracts the first k rows of data from the sub-table generated by the sub-table generation module, combines the table status and table description extracted in the spreadsheet file upload and extraction module with the system prompt of the planner module, and provides them to the planner module based on the large language model in the agent;

[0036] The code generation and execution module generates Python code through the planner module according to the information provided by the data extraction module, and executes it in the interpreter to obtain the code execution result;

[0037] The judgment module makes a judgment based on the code execution results obtained in the code generation and execution modules. If the execution is successful, the code is executed on the target spreadsheet, and the spreadsheet status is updated to proceed to the next step of planning. If the execution fails, the retriever module in the agent retrieves similar code fragments to assist the planner module in regenerating the code until the code is successfully executed.

[0038] The loop module repeats the operations from the spreadsheet file upload and extraction module to the judgment module until the task is completed and saves the spreadsheet after the operation.

[0039] Compared with the prior art, the beneficial effects of the present invention are: implementing the information provider module and the planner module based on the large language model, and retrieving the sample code of the spreadsheet file operation from the code base through the retriever module to assist the planner module in generating Python code, thereby improving its robustness and accuracy. On this basis, the information provider module, the planner module and the retriever module are coordinated through the steps of extracting table information, generating sub-tables, extracting data, generating and executing codes, judging the execution status and looping. Therefore, the present invention provides a new method and system for automated operation and reasoning of spreadsheets based on a large language model, which can cope with more complex spreadsheet operation and reasoning tasks, including coping with reasoning challenges such as fuzzy expressions and multi-step reasoning, and effectively improves the ability and efficiency of automatically solving spreadsheet operation and reasoning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the working process of an embodiment of the present invention;

[0041] Figure 2 It is a data set characteristic and evaluation process diagram of an embodiment of the present invention;

[0042] Figure 3 It is a schematic diagram of an application scenario for guiding the work of an intelligent agent according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In the description of the present invention, if it involves a description of orientation, for example, the orientation or position relationship indicated by "upper", "lower", "front", "back", "left", "right", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. If a certain technical feature is referred to as "set", "fixed", "connected", or "installed" on another technical feature, it can be directly set, fixed, or connected to the other technical feature, or it can be indirectly set, fixed, connected, or installed on the other technical feature.

[0044] In the description of the present invention, if "several" is involved, it means more than one; if "multiple" is involved, it means more than two; if "greater than", "less than", or "exceed" is involved, it should be understood as not including the number itself; if "above", "below", or "within" is involved, it should be understood as including the number itself. If "first", "second", etc. are involved, they should be understood as being used only to distinguish the names of the same or similar technical features, and should not be understood as implying / indicating the relative importance of the technical features, the number of the technical features, or the order of the technical features.

[0045] The preferred embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0046] like Figures 1 to 3 As shown, this embodiment provides an electronic spreadsheet automatic operation and reasoning method based on a large language model, comprising the following steps:

[0047] Step S1, uploading an electronic form file and a user instruction, and extracting relevant information from the electronic form file, the relevant information including form status, form description, system prompts of the information provider module, and system prompts of the planner module;

[0048] Step S2, providing the table state, table description and system prompt of the information provider module extracted in step S1 to the information provider module implemented based on the large language model in the intelligent agent;

[0049] Step S3, generating an SQL query statement based on the information provided in step S2, and executing it in the database to generate a subtable corresponding to the uploaded electronic form;

[0050] Step S4, extracting the first k rows of data from the subtable generated in step S3, combining the table state, table description and system prompt of the planner module extracted in step S1, and providing them to the planner module implemented based on the large language model in the agent;

[0051] Step S5, generating Python code through the planner module according to the information provided in step S4, and executing it in the interpreter to obtain the code execution result;

[0052] Step S6, judging according to the code execution result obtained in step S5, if the execution is successful, the code is executed on the target electronic form, and the electronic form state is updated to proceed to the next step of planning; if the execution fails, the retriever module in the agent retrieves similar code fragments to assist the planner module in regenerating the code until the code execution is successful;

[0053] Step S7, repeating steps S1 to S6 until the task is completed, and saving the electronic spreadsheet after the operation.

[0054] It should be noted that, unlike the prior art, the information provider module (Informer module) of this embodiment generates and executes SQL statements through steps S1-S3 on the premise of understanding the user's instructions, and can submit the subtable corresponding to the spreadsheet file to the planner module (Planner module) as evidence highly related to the instructions, without having to pass the entire spreadsheet to the Planner, thereby effectively reducing the Planner's reasoning burden and improving its reasoning ability and efficiency. On this basis, through other steps of subsequent collaborative work, the Planner generates operation instructions through Python code, and cooperates with the Retriever module (Retriever module) to make better code corrections, thereby further improving its reasoning ability.

[0055] The intelligent agent described in this embodiment includes an information provider module, a planner module and a retriever module. The information provider module is also called an Informer module, abbreviated as Informer; the planner module is also called a Planner module, abbreviated as Planner; the retriever module is also called a Retriever module, abbreviated as Retriever.

[0056] Among them, the information provider module and the planner module are implemented based on a large language model; the information provider module is used to parse the task requirements and generate high-quality, task-related SQL query statements to extract sub-table information and send it to the planner module in combination with some relevant information of the spreadsheet file, so that the planner module can better understand the spreadsheet without reading all the table data, and reduce the consumption of tokens while ensuring a full understanding of the table. The planner module is used to translate the conceptual understanding of the task semantics into Python code to implement code execution and operation, so as to accurately operate the spreadsheet; Python code is a series of instructions written in the Python programming language, which can be executed by the Python interpreter to complete a specific task or program. The retriever module is used to retrieve sample code for spreadsheet file operations from the code library to assist the planner module in generating Python code and improve the robustness and accuracy of the code generated by the planner module.

[0057] Before the present application is specifically described in conjunction with the embodiments, the data set used in the present embodiment is first introduced. Existing data sets, such as the Sheet Copilot data set and the Instruct GPT data set, are designed to achieve more precise spreadsheet manipulation, which does not meet the actual needs of the present application.

[0058] Since this application is intended to meet more realistic and complex task requirements, it is necessary to solve the challenges involved, such as multi-step reasoning and fuzzy expressions. Therefore, in order to better illustrate the technical effects of this application in conjunction with the embodiments, first, in this embodiment, a Figure 2 The more realistic and complex dataset shown is called the Sheet RM dataset, and one of the tasks is used as an example. Figure 2 Parts (a)-(d) of the dataset show the characteristics of the Sheet RM dataset, including: multiple categories, including 5 major categories of spreadsheet operations, 36 minor categories, and 4 reasoning challenges. The 5 major categories and 36 minor categories are detailed in Figure 2 (a) Worksheet management, value processing, content aggregation, graph design, format adjustment in multiple categories and 36 subcategories in the outer circle, 4 reasoning challenges including complex computational logic, fuzzy requirements, inconsistent data formats and information extraction, to test the operation and reasoning ability of the agent. (b) Operations that rely on reasoning, that is, the tasks in the Sheet RM dataset include operations that require multi-step reasoning on the content of the spreadsheet; (c) Long sequences, that is, each complete task includes multiple different subtasks; (d) Process evaluation, that is, an end-to-end automated evaluation process is implemented to more flexibly and comprehensively evaluate the completion of the task.

[0059] by Figure 3 Taking the task shown in as an example, in step S1 of this embodiment, the uploaded spreadsheet file only includes a table of "Electronic Products", and the task instruction is "Highlight the best-selling computer accessories in yellow, and draw a pie chart reflecting the sales of each product". The relevant information extracted from the uploaded spreadsheet file includes the table status, table description, system prompt of the information provider and system prompt of the planner module; system prompt is also called system prompt, that is, SystemPrompt, which usually refers to the message or instruction displayed by the computer system or software program to guide the user to perform operations or provide information. The table status records the number of rows and columns, the name of each column (such as "product", "unit price", etc.) and the data type (such as "str", "int") of each table data in the uploaded spreadsheet; the table description records a brief description of the content or subject of the uploaded spreadsheet, given in natural language. In this embodiment, Figure 3 For example, “This worksheet records the sales of electronic products.” User instructions refer to the user's operating instructions for the electronic spreadsheet file.

[0060] The implementation process of extracting relevant information from the electronic spreadsheet file in step S1 of this embodiment includes the following sub-steps:

[0061] Step S101, the user selects and uploads an electronic spreadsheet file through the interface. After receiving the electronic spreadsheet file, first verify whether the electronic spreadsheet file format is a pre-set supported electronic spreadsheet format, for example, whether it is a pre-set electronic spreadsheet format such as xlsx or .xls. If the format is incorrect, the user is prompted to re-upload, and the process jumps to step S102 until the electronic spreadsheet file format verification passes;

[0062] Step S102, parsing the uploaded spreadsheet file, using the pandas data processing and analysis library of the scripting language Python to read the data in the spreadsheet file and converting it into a data frame DataFrame format;

[0063] Step S103, according to the obtained data frame, extract and record the corresponding table status, table description, system prompt of the information provider module and system prompt of the planner module, the table status is used to record the number of rows, columns, names of each column and data type of each table data in the uploaded electronic spreadsheet file, and the table status is extracted through the object attribute acquisition function of the scripting language Python; the table description is used to record the content or theme summary of the uploaded electronic spreadsheet file, and the table description is extracted through the uploaded natural language; the system prompt of the information provider and the system prompt of the planner module are used to prompt the identification information of the information provider and the planner module respectively, and are extracted through the provided hard-coded form.

[0064] The specific implementation process of step S102 in this embodiment preferably includes the following sub-steps:

[0065] Step S1021, installing the pandas data processing and analysis library, also known as the pandas library;

[0066] Step S1022, using the read_excel function of the pandas data processing and analysis library to read the spreadsheet file, and converting it into a core data structure DataFrame according to the worksheet name or index of the spreadsheet file;

[0067] Step S1023, processing the converted data, including missing value processing and deleting duplicate rows, etc.; filling the missing positions with 0 to achieve placeholders and avoid subsequent information errors;

[0068] Step S1024, perform data query by any one or more of the instructions head(), descriptive statistics instructions describe(), and data information instructions info(); wherein, the instruction head() is used to view the first few rows of the core data structure DataFrame, and the first 5 rows are displayed by default; the descriptive statistics instruction describe() is used to generate descriptive statistics information of the numerical columns in the core data structure DataFrame, including the mean, median, maximum value, minimum value, standard deviation, etc.; the data information instruction info() is used to display the basic information of the core data structure DataFrame, including the name and data type of the index, the data type of each column, the number of non-null values, etc.;

[0069] Step S1025, use conversion instructions to convert the queried data, the conversion instructions include at least one of query(), loc[], iloc[] and filter(); wherein, the query() instruction refers to an instruction to filter data based on a string expression, loc[] refers to an instruction to select data based on a label; iloc[] refers to an instruction to select data based on an integer position; filter() refers to an instruction to select a column based on a string match of the column name.

[0070] In step S2 of this embodiment, the information provider module implemented based on the large language model refers to an informer used to parse task requirements and generate SQL query statements related to the task. The large language model refers to an intelligent model obtained after training with a large amount of text data, which is based on the transformer architecture through deep learning and neural network technology so as to be able to learn the language model. The implementation process of the information provider module includes the following sub-steps:

[0071] Step S201, parsing the query requirements about the spreadsheet file through a natural language processing module (such as a deep learning model: BERT or GPT, etc.), wherein the parsing of the natural language processing module includes entity recognition, intent recognition and semantic understanding, and is used to convert the table structure, some rows, columns and cell data of the spreadsheet into text form, which is used as the key parameter of the SQL query;

[0072] Step S202, the generation module composed of the large language model generates corresponding SQL query statements based on the query requirements of the parsed electronic spreadsheet file and the table data in text form through internal knowledge to ensure the query performance.

[0073] In step S3 described in this embodiment, the specific process of generating an SQL query statement based on the information provided in step S2 includes: storing all table data of the current state of the spreadsheet file in the database, integrating the information provided in step S2 into the command symbol Prompt and inputting it into the large language model, and generating an SQL query statement through the large language model; the generated SQL query statement is executed in the database and a result is obtained, and the result is used as a subtable corresponding to the uploaded spreadsheet.

[0074] Specifically, the table data in the target electronic form (such as the uploaded electronic form) is extracted and stored in a lightweight database. In this embodiment, the default is the SQLite database (i.e., an embedded database for relational data management). The goal of the information provider module is to query and obtain the part of the table data and its task instructions that are most relevant to the current task step from the complete table, so that the planner can better understand the table data without consuming a large number of tokens, thereby completing the task more efficiently. To achieve this goal, assuming that the current step is step t, the information provider module first integrates the input obtained, including the system prompt P I , task instruction I, table status s t , and the sequence of steps A produced by the planner module in the previous planning history t-1 =(a1,…,a t-1 ). Then, the information provider generates the SQL query statement q t =Informer(q t |P I ,I,A t-1 ,s t ). That is, in the implementation process of generating SQL query statements through a large language model in this embodiment, first enter the system prompt P I , task instruction I, table status s t and the sequence of steps A produced by the planner module in the previous planning history t-1 , and then use the SQL query statement q t =Informer(q t |P I ,I,A t-1 ,s t ), where Informer(q t |*) represents the generated SQL query statement, P I Indicates the system prompt of the information provider module, I indicates the task instruction, s t Indicates the table status; A t-1represents the reasoning trajectory of the planner module, which enables the information provider module to generate SQL query statements that are more relevant and more stable to the task. It should be noted that when the planner module has not started planning, the step sequence A t-1 Finally, the SQL query statement generated by the information provider is executed in the above SQLite database. t , get the subtable data e t .

[0075] In step S4 of this embodiment, the process of extracting the first k rows of data from the subtable generated in step S3 includes the following sub-steps:

[0076] Step S401, using the pandas data processing and analysis library of the scripting language Python to read the content of the spreadsheet file and convert it into a data frame format, preferably including reading a CSV file through a read_csv() function, reading an Excel file through a read_excel() function, reading data from a SQL database through a read_sql() function, and reading a file in JSON format through a read_json() function. Through the above functions, data can be automatically converted into a DataFrame object while reading. DataFrame is the main data structure in the pandas data processing and analysis library for storing and operating structured data;

[0077] Step S402, extract the first k rows of data of the sub-table through the slice operation instruction df[:k] of the pandas data processing and analysis library, for example, input the instruction subset_df=df[:k], where k is a natural number, indicating the number of rows to be extracted.

[0078] In step S5 described in this embodiment, the specific process of the planner module generating Python code includes: integrating the information provided in step S4 into the command symbol Prompt and inputting it into the large language model, and generating Python code through the large language model; the generated Python code is executed in the virtual sandbox environment of the interpreter, and the generated result is binary information, which is used to represent correct or incorrect respectively.

[0079] Specifically, assuming that the current step is t, given the task instruction I and the system prompt P P , table description D and table status s t , the planner module generates action a t =Planner(P P ,I,D,s t ,h t-1 ), where h t-1 is the planning history aggregated from the previous steps. Action a tContains the planner module's ideas and solutions for the current step, where the solution is expressed in Python code. Specifically, the generated Python code is executed in a virtual sandbox environment to evaluate its correctness for the target spreadsheet operation and obtain feedback. t =Sandbox(a t ).

[0080] Preferably, in step S6 of this embodiment, if the execution is successful, action a is executed on the target electronic form. t The Python code contained in the table status s t Updated to t+1 , and update the planning history to h t =(h t-1 ,o t ,a t ) in order to record and save historical data, spreadsheet operation feedback, and execution actions in the planning history, so as to facilitate the dynamic self-learning process to update and gradually improve the intelligent agent based on the large language model in real time.

[0081] In step S6 of this embodiment, similar code snippets are retrieved by the retriever module in the agent (i.e., the top k most similar code snippets are retrieved). ), to assist the planner module in regenerating the code as follows:

[0082] Step S601: Execute the wrong code through the Embedding vector mapping model Convert to vector as a query vector;

[0083] Step S602: Input the query vector into the code base Retrieve the top k most similar code snippets

[0084] Step S603: convert the above k code snippets As additional information for the planner module, return to step S5 to have the planner module regenerate Python code.

[0085] Preferably, in step S6, this embodiment first determines whether the execution result in step S5 is successful. If so, it directly jumps and performs the next step of planning. If not, it jumps to step S601 and retrieves similar code fragments through the retriever module to assist the planner module in regenerating the code.

[0086] In this embodiment, step S601 will execute the wrong code Convert to vector The implementation process is as follows:

[0087] Step S6011, use regular expressions to remove codes Comments and extra whitespace in strings;

[0088] Step S6012: Convert the code into a vector by using the encode numerical conversion method of the Embedding vector mapping model.

[0089] The code library in step S602 of this embodiment The construction process includes: collecting high-quality codes from the Internet, writing sample codes for different spreadsheet operations, and organizing the sample codes into files; using the Milvus vector database as the code library, traversing the code files using the sliding window technology, and obtaining the traversed code snippet collection as the code library The code data foundation is then used; then the Embedding vector mapping model is used to convert the code corresponding to the code snippet set into vectors and store them in the code library middle.

[0090] More specifically, for the code library in this embodiment The construction process of The code in the example can be derived from the open source projects on the GitHub website and the official examples listed in external Python libraries such as openpyxl and pandas. In order to ensure the representativeness of the code, this embodiment preferably follows the following Figure 2 The codes are organized and adjusted based on the operations covered by the SheetRM benchmark shown in the figure, and then abstracted to achieve universality. For operations for which there is no ready-made sample code, this embodiment further collects more code from the above-mentioned code source or further writes this part of the code to ensure that all operations can be covered. These organized codes provide data-based guidance. Although the data in the real case is related to the task scenario, since this embodiment abstracts and encapsulates the collected code and only provides the part of the code on how to perform specific operations on the spreadsheet, this will not cause excessive misjudgment of the large language model.

[0091] In this embodiment, in step S602, the top k most similar code snippets are retrieved. The following formula is preferably used: in k represents the number of code snippets, and sim(·) represents the cosine similarity between two vectors, that is: Represents the i-th code in the code base; It is a universal quantifier, which is used to indicate that all elements are true; Represents the jth code in the code base; Representation Code Base The top k most similar code snippets retrieved from Indicates that the model ε will execute the wrong code through the Embedding vector mapping The transformed vector representation; and Similarly, It represents the vector representation after the execution error i-th code segment is converted through the Embedding vector mapping model ε. It represents the vector representation after converting the j-th code segment with execution error through the Embedding vector mapping model ε; i and j are natural numbers, which are used to represent the ordinal numbers of the code fragments respectively.

[0092] It should be noted that step S601 described in this embodiment preferably uses Open AI's text-embedding-ada-002 model as the Embedding vector mapping model ε. The text-embedding-ada-002 model is a text embedding tool, which is used as an Embedding vector mapping model in this embodiment to realize the conversion from code snippets to vectors. It is suitable for processing text data in a variety of different languages, and provides an API interface to obtain embedding processing, which can make the retrieval model of similar code snippets simpler and more efficient. The specific process is preferably: first, the meaning and context of the code snippet are parsed through the text embedding tool of the text-embedding-ada-002 model, and the semantic information therein is extracted through the deep learning model, so as to convert the code snippet into a vector, for example, through the instruction def get_embedding(text,model="text-embedding-ada-002"), where def get_embedding() refers to a function named get_embedding, text represents the code snippet to be converted, and model="text-embedding-ada-002" means that the code snippet is input into the specified text-embedding-ada-002 deep learning model model for meaning and context parsing, and then the embedding vector of the text is obtained through its API interface, such as obtaining the embedding vector through the get_embedding function, and using this embedding vector as the query vector.

[0093] More preferably, combined with Figure 3In step S603 of this embodiment, the process of regenerating Python code by the planner module occurs in the second planning step: the error code generated by the planner module is evaluated in the virtual sandbox environment, and the feedback of "execution failure" is obtained and the specific failure reason is provided. After the two stages of steps S601-S602, the top k most similar code fragments are Arrange in descending order according to the similarity score and integrate into the input described in step S5; regenerate the action according to the process of step S5, and the new action is Specifically, in this embodiment, only the top three most similar code snippets are retrieved by default.

[0094] In order to enable those skilled in the art to better understand the present application and prove the effectiveness of the present application, this embodiment uses the SheetRM evaluation set (i.e., the benchmark test set of generalist agents) as a research case. The experimental scheme can be expanded to a larger scope. The experiment is divided into two parts. The first part is the application of the method, that is, using the method to solve 201 complete tasks in the generalist agent SheetRM; the second part is the result evaluation, using Figure 2 The automated evaluation method illustrated in the figure evaluates the technical solution generated in the first part, and uses a variety of evaluation indicators to give a scoring result to measure the effectiveness of this application.

[0095] This experiment selected a variety of large language models as the basis, including but not limited to the GPT-3.5 model, the GPT-4 model, the Claude 3 model, the Qwen-1.5 model and the LLaMA3 model. It should be noted that those skilled in the art can select different large language models and set different hyperparameters for different task scenarios, such as the number of retrieved code snippets k, without having to refer to the values ​​described in this embodiment. The parameters provided in this embodiment are only used as test cases and should not be understood as limitations on this application.

[0096] When evaluating the solution, this embodiment uses three evaluation indicators, namely, single task pass rate (Pass@1), single subtask pass rate (SubPass@1), and single execution success rate (Exec@1), to measure the effectiveness of the method. These three indicators are expressed by the following formulas: as well as Where T represents the total number of tasks, T c Indicates the number of tasks successfully completed, S c,i represents the number of subtasks successfully completed in the i-th task, S i represents the number of subtasks in the i-th task, E cIndicates the number of tasks that are correctly executed. Table 1 below shows the three evaluation index values ​​of this embodiment on different large language models.

[0097] Table 1 Results of this application on the SheetRM evaluation set when using different large language models

[0098] Evaluation Metrics GPT-3.5 GPT-4 Claude 3 Qwen-1.5 LLaMA 3 Pass@1 30.8 42.3 25.4 7.0 13.4 SubPass@1 69.1 76.3 65.0 20.4 32.7 Exec@1 92.5 88.6 92.0 48.8 68.7

[0099] As can be seen from Table 1 above, the present invention shows significant improvement on closed-source large language models such as GPT and Claude, which shows that the present invention can further improve the operation and reasoning capabilities of large language models that are already excellent enough for spreadsheets, and can cope with reasoning challenges such as fuzzy expressions and multi-step reasoning of complex spreadsheets to solve more complex spreadsheet reasoning tasks. In addition, despite the use of small models such as Qwen and LLaMA, the present invention can still effectively improve their effects, indicating that the present invention can stimulate the operation and reasoning capabilities of smaller models for spreadsheets to meet different spreadsheet automation operations and reasoning requirements.

[0100] In practical applications, those skilled in the art may also select other indicators to evaluate the performance of the present application. When the indicators of the technical solution provided by the present application on the evaluation set reach the preset threshold, it can be considered that the method can provide an effective solution, and the preset threshold can be adjusted according to actual conditions and needs.

[0101] This embodiment also provides an electronic form automatic operation and reasoning system based on a large language model, which adopts the electronic form automatic operation and reasoning method based on a large language model as described above, and includes:

[0102] An electronic spreadsheet file upload and extraction module, which uploads electronic spreadsheet files and user instructions, and extracts relevant information from the electronic spreadsheet files, the relevant information including form status, form description, system prompts of the information provider module, and system prompts of the planner module;

[0103] An input module, which provides the table status, table description and system prompt of the information provider module extracted from the spreadsheet file upload and extraction module to the information provider module implemented based on the large language model in the intelligent agent;

[0104] The sub-table generation module generates SQL query statements based on the information provided by the input module and executes them in the database to generate sub-tables corresponding to the uploaded electronic forms;

[0105] The data extraction module extracts the first k rows of data from the sub-table generated by the sub-table generation module, combines the table status and table description extracted in the spreadsheet file upload and extraction module with the system prompt of the planner module, and provides them to the planner module based on the large language model in the agent;

[0106] The code generation and execution module generates Python code through the planner module according to the information provided by the data extraction module, and executes it in the interpreter to obtain the code execution result;

[0107] The judgment module makes a judgment based on the code execution results obtained in the code generation and execution modules. If the execution is successful, the code is executed on the target spreadsheet, and the spreadsheet status is updated to proceed to the next step of planning. If the execution fails, the retriever module in the agent retrieves similar code fragments to assist the planner module in regenerating the code until the code is successfully executed.

[0108] The loop module repeats the operations from the spreadsheet file upload and extraction module to the judgment module until the task is completed and saves the spreadsheet after the operation.

[0109] In summary, this embodiment implements the information provider module and the planner module based on the large language model, and retrieves the sample code of the spreadsheet file operation from the code base through the retriever module to assist the planner module in generating Python code to improve its robustness and accuracy. On this basis, the information provider module, the planner module and the retriever module are also realized through the steps of table information extraction, sub-table generation, data extraction, code generation and execution, execution status judgment and looping. Therefore, the present invention provides a new method and system for automatic operation and reasoning of spreadsheets based on a large language model, which can cope with more complex spreadsheet operation and reasoning tasks, including coping with reasoning challenges such as fuzzy expressions and multi-step reasoning, and effectively improves the ability and efficiency of automatically solving spreadsheet operation and reasoning tasks.

[0110] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for automated operation and reasoning of electronic spreadsheets based on a large language model, characterized in that: The following steps are involved: Step S1, uploading an electronic form file and a user instruction, and extracting relevant information from the electronic form file, the relevant information including form status, form description, system prompts of the information provider module, and system prompts of the planner module; Step S2, providing the table state, table description and system prompt of the information provider module extracted in step S1 to the information provider module implemented based on the large language model in the intelligent agent; Step S3, generating an SQL query statement based on the information provided in step S2, and executing it in the database to generate a subtable corresponding to the uploaded electronic form; Step S4, extracting the first k rows of data from the subtable generated in step S3, combining the table state, table description and system prompt of the planner module extracted in step S1, and providing them to the planner module implemented based on the large language model in the agent; Step S5, generating Python code through the planner module according to the information provided in step S4, and executing it in the interpreter to obtain the code execution result; Step S6, judging according to the code execution result obtained in step S5, if the execution is successful, executing the code on the target electronic form, updating the electronic form status, and performing the next step of planning; If the execution fails, similar code snippets are retrieved through the retriever module in the agent to assist the planner module in regenerating the code until the code is successfully executed; Step S7, repeating steps S1 to S6 until the task is completed, and saving the electronic spreadsheet after the operation.

2. The electronic spreadsheet automated operation and reasoning method based on a large language model according to claim 1, characterized in that: The intelligent agent includes an information provider module, a planner module and a retriever module, wherein the information provider module and the planner module are implemented based on a large language model; the information provider module is used to parse task requirements and generate SQL query statements related to the task so as to extract sub-table information and send it to the planner module in combination with some relevant information of the spreadsheet file; the planner module is used to translate the task semantics into Python code so as to implement code execution and operation; the retriever module is used to retrieve sample codes for spreadsheet file operations from a code library to assist the planner module in generating Python code.

3. The electronic spreadsheet automated operation and reasoning method based on a large language model according to claim 1 or 2, characterized in that: The implementation process of extracting relevant information from the electronic spreadsheet file in step S1 includes the following sub-steps: Step S101, the user selects and uploads an electronic spreadsheet file through the interface. After receiving the electronic spreadsheet file, first verify whether the electronic spreadsheet file format is a pre-set supported electronic spreadsheet format. If the format is incorrect, prompt the user to re-upload it. Jump to step S102 until the electronic spreadsheet file format verification passes; Step S102, parsing the uploaded spreadsheet file, using the pandas data processing and analysis library of the scripting language Python to read the data in the spreadsheet file and converting it into a data frame DataFrame format; Step S103, according to the obtained data frame, extract and record the corresponding table status, table description, system prompt of the information provider module and system prompt of the planner module, the table status is used to record the number of rows, columns, names of each column and data type of each table data in the uploaded electronic spreadsheet file, and the table status is extracted through the object attribute acquisition function of the scripting language Python; the table description is used to record the content or theme summary of the uploaded electronic spreadsheet file, and the table description is extracted through the uploaded natural language; the system prompt of the information provider and the system prompt of the planner module are used to prompt the identification information of the information provider and the planner module respectively, and are extracted through the provided hard-coded form.

4. The electronic spreadsheet automated operation and reasoning method based on a large language model according to claim 1 or 2, characterized in that: In step S3, the specific process of generating an SQL query statement based on the information provided in step S2 includes: storing all table data of the current state of the spreadsheet file in the database, integrating the information provided in step S2 into the command symbol Prompt and inputting it into the large language model, and generating an SQL query statement through the large language model; the generated SQL query statement is executed in the database and a result is obtained, and the result is used as a subtable corresponding to the uploaded spreadsheet.

5. The electronic spreadsheet automated operation and reasoning method based on a large language model according to claim 4, characterized in that: Generate SQL query statements through large language models using statement q t =Informer(q t |P I ,I,A t-1 ,s t ), where Informer(q t |*) represents the generated SQL query statement, P I Indicates the system prompt of the information provider module, I indicates the task instruction, A t-1 represents the reasoning trajectory of the planner module, s t Indicates the table status.

6. The electronic spreadsheet automated operation and reasoning method based on a large language model according to claim 1 or 2, characterized in that: In step S4, the implementation process of extracting the first k rows of data from the subtable generated in step S3 includes the following sub-steps: Step S401, using the pandas data processing and analysis library of the scripting language Python to read the content of the spreadsheet file and convert it into a data frame format; Step S402: extract the first k rows of data in the sub-table through the slice operation instruction df[:k] of the pandas data processing and analysis library.

7. The electronic spreadsheet automated operation and reasoning method based on a large language model according to claim 1 or 2, characterized in that: In step S5, the specific process of the planner module generating Python code includes: integrating the information provided in step S4 into the command symbol Prompt and inputting it into the large language model, and generating Python code through the large language model; the generated Python code is executed in the virtual sandbox environment of the interpreter, and the generated result is binary information, which is used to represent correct or incorrect respectively.

8. The electronic spreadsheet automated operation and reasoning method based on a large language model according to claim 1 or 2, characterized in that: In step S6, the implementation process of retrieving similar code snippets by the retriever module in the agent to assist the planner module in regenerating the code is as follows: Step S601: Execute the wrong code through the Embedding vector mapping model Convert to vector as a query vector; Step S602: Input the query vector into the code base Retrieve the top k most similar code snippets Step S603: convert the above k code snippets As additional information for the planner module, return to step S5 to have the planner module regenerate Python code.

9. The electronic spreadsheet automated operation and reasoning method based on a large language model according to claim 7, characterized in that: The step S601 will execute the wrong code Convert to vector The implementation process is as follows: Step S6011, use regular expressions to remove codes Comments and extra whitespace in strings; Step S6012: Convert the code into a vector by using the encode numerical conversion method of the Embedding vector mapping model.

10. An electronic spreadsheet automation operation and reasoning system based on a large language model, characterized in that: The electronic form automated operation and reasoning method based on a large language model as claimed in any one of claims 1 to 9 is adopted, and includes: An electronic spreadsheet file upload and extraction module, which uploads electronic spreadsheet files and user instructions, and extracts relevant information from the electronic spreadsheet files, the relevant information including form status, form description, system prompts of the information provider module, and system prompts of the planner module; An input module, which provides the table status, table description and system prompt of the information provider module extracted from the spreadsheet file upload and extraction module to the information provider module implemented based on the large language model in the intelligent agent; The sub-table generation module generates SQL query statements based on the information provided by the input module and executes them in the database to generate sub-tables corresponding to the uploaded electronic forms; The data extraction module extracts the first k rows of data from the sub-table generated by the sub-table generation module, combines the table status and table description extracted in the spreadsheet file upload and extraction module with the system prompt of the planner module, and provides them to the planner module based on the large language model in the agent; The code generation and execution module generates Python code through the planner module according to the information provided by the data extraction module, and executes it in the interpreter to obtain the code execution result; The judgment module makes a judgment based on the code execution results obtained in the code generation and execution modules. If the execution is successful, the code is executed on the target spreadsheet, and the spreadsheet status is updated to proceed to the next step of planning. If the execution fails, the retriever module in the agent retrieves similar code fragments to assist the planner module in regenerating the code until the code is successfully executed. The loop module repeats the operations from the spreadsheet file upload and extraction module to the judgment module until the task is completed and saves the spreadsheet after the operation.

Images