Workflow engine data analysis method and system based on large model driving

Abstract

The invention discloses a workflow engine data analysis method and system based on large model driving. The method comprises the following steps that a natural language requirement input by a user is obtained; based on natural language requirements, keywords are extracted and subjected to importance sorting, and sorted user requirement keywords are obtained; querying a corresponding preset knowledge base based on the user demand keyword, and converting the preset knowledge base into a to-be-executed query statement; building prompt words according to the user demand keywords and the query statements, inputting the prompt words into a large language model for processing, and converting the prompt words into workflows to be executed; converting a to-be-executed workflow step into workflow node information; converting the workflow node information into a standard business process model; and loading the model into the workflow engine, executing the model, and returning an execution result. By introducing a large model driven workflow construction mode, the intelligent level of workflow construction is improved, the user operation difficulty is reduced, and the efficiency and accuracy of data analysis are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and data analysis, and more specifically to a method and system for constructing data analysis of a workflow engine driven by a large model. Background Art

[0002] With the in-depth development of the era of big data and big models, the level of digitization and intelligence is constantly improving, and it is possible to implement big data analysis and intelligent analysis of business processes through natural language. Although traditional big data analysis tools provide drag-and-drop selection functions, allowing users to define data sources, data processing, and visualization without writing complex codes, there are still some problems.

[0003] 1) High threshold of professional knowledge and programming requirements: Although existing tools reduce the need for code writing, users still need to have professional business knowledge and certain programming knowledge. For non-relevant technical personnel, the learning and use costs are still high.

[0004] 2) Lack of intelligence and automation: Existing tools mainly rely on manual settings and configurations by users in terms of data processing and calculation, lacking support for intelligence and automation, resulting in low efficiency.

[0005] 3) Limited data utilization and difficult integration: Data terminals are basically data from data providers. ETL tools provide processed internal enterprise data, but users still cannot effectively use this data. If users want to use their own data for data applications, they will face secondary development based on data terminals, and there are problems such as long development cycles and high development costs. In addition, the integration of different data sources and formats is still a challenge, especially when faced with a large amount of heterogeneous data, the processing complexity increases significantly.

[0006] 4) Poor maintenance and scalability: As business needs change, traditional tools have high adjustment and maintenance costs and are difficult to quickly adapt to new needs or technology updates.

[0007] 5) Insufficient analytical capabilities: Traditional tools have limited capabilities in intelligent analysis and lack deep analysis functions based on big data and machine learning algorithms, and are unable to fully tap the potential value of data.

[0008] 6) After the data collection, processing, analysis and presentation process is completed, the task of data analysis is far from over. Simply listing the data is not enough to bring out its value. The key is to analyze the data in depth, extract insightful analysis conclusions, and compile a detailed data analysis report based on them. This process also requires a lot of time and effort.

[0009] Therefore, using large models to build workflows and use them in big data analysis processes to achieve efficient, flexible and intelligent data analysis is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0010] In view of this, the present invention provides a workflow engine data analysis method and system based on a big model driven, which uses advanced big language model technology to automatically build business workflow tasks according to the natural language description provided by the user.

[0011] In order to achieve the above object, the present invention adopts the following technical solution:

[0012] In a first aspect, the present invention proposes a workflow engine data analysis method based on a large model drive, comprising the following steps:

[0013] S1. Obtain natural language requirements input by users;

[0014] S2. Based on the natural language demand, extract keywords and sort them by importance to obtain sorted user demand keywords;

[0015] S3, querying a corresponding preset knowledge base based on the user demand keyword, and converting the user demand keyword into a query statement to be executed;

[0016] S4, constructing prompt words according to the user demand keywords and query statements, and inputting them into the large language model for processing, and converting them into workflow steps to be executed;

[0017] S5, converting the workflow steps to be executed into workflow node information; converting the workflow node information into a standard business process model;

[0018] S6. Load the standard business process model into the workflow engine and execute it, and return the execution result.

[0019] In a second aspect, the present invention proposes a workflow engine data analysis system based on a large model drive, comprising the following modules:

[0020] User input module: used to obtain natural language input requirements from users;

[0021] Preprocessing module: extract keywords based on the natural language demand and sort them by importance to obtain sorted user demand keywords;

[0022] Large language model processing module: used to query the corresponding preset knowledge base based on the user demand keywords, and convert the user demand keywords into query statements to be executed;

[0023] It is also used to construct prompt words according to the user's demand keywords and query statements, and input them into the large language model for processing and convert them into workflow steps to be executed;

[0024] Workflow construction module: used to convert the workflow steps to be executed into workflow node information; convert the workflow node information into a standard business process model;

[0025] Tools and plug-in modules: used to load the standard business process model into the workflow engine and execute it, and return the execution result.

[0026] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a workflow engine data analysis method and system based on large model drive, which has the following beneficial effects:

[0027] 1. Support users to input requirements in natural language, which lowers the usage threshold and improves user experience;

[0028] 2. Provides a variety of node types, such as plug-in nodes, large model nodes, workflow nodes, etc., to meet diverse data analysis needs;

[0029] 3. By introducing a large model-driven workflow construction method, the intelligence level of workflow construction is improved, the difficulty of user operation is reduced, and the efficiency and accuracy of data analysis are significantly improved.

[0030] 4. Realize dynamic adjustment of workflow and real-time feedback, ensuring flexibility and efficiency of data analysis;

[0031] 5. By integrating big models, natural language processing, knowledge base query and big data query technology, a comprehensive, efficient and user-friendly data analysis platform is built. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0033] Figure 1 A flow chart of a workflow engine data analysis method based on large model drive provided in an embodiment of the present invention.

[0034] Figure 2 A schematic diagram of function calls in a workflow provided by an embodiment of the present invention.

[0035] Figure 3A schematic diagram of a workflow engine data analysis system driven by a large model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Workflow, originated from the field of production organization and office automation, refers to the automation of part or all of a business process in a computer application environment. The present invention utilizes workflow to decompose complex tasks into well-defined tasks or roles (nodes), and executes the tasks in these steps according to certain rules and processes, thereby reducing system complexity, reducing dependence on prompt word technology and model reasoning capabilities, improving the performance of LLM applications in complex tasks, and enhancing the interpretability, stability and fault tolerance of the system.

[0038] Embodiment 1

[0039] The embodiment of the present invention discloses a workflow engine data analysis method based on a large model drive, referring to Figure 1 As shown, the following steps S1 to S6 are included:

[0040] S1. Obtain natural language requirements input by users;

[0041] Receive data analysis requirements input by users in natural language.

[0042] For example, when a user inputs "analyze the price trend of steel in Shanghai and predict the next three months", the large language model receives the instruction and prepares for subsequent processing.

[0043] S2. Based on natural language requirements, extract keywords and sort them by importance to obtain sorted user requirement keywords;

[0044] Using natural language processing (NLP) technology, keywords are extracted from user input and sorted according to their semantic importance.

[0045] In this embodiment, prompt words are first constructed by natural language, and prompt word templates are used to generate JSON, a lightweight data exchange format, in accordance with the prompt word framework.

[0046] The prompt words are as follows:

[0047] #Role

[0048] You are an experienced requirements analyst who can identify the key points in the user's requirements.

[0049] #Follow

[0050] Return the keywords in the requirements in the form of a JSON array, grouped according to different tasks, and sorted by importance.

[0051] #Requirements

[0052] I want to analyze the steel price trend in the Shanghai market recently and predict the trend in the next three months. After sorting, output JSON:

[0053] [{"keyword":["Shanghai market","Steel price trend","Recently"],"importance":3},{"keyword":["Shanghai market","Steel price trend","Next three months","Prediction"],"importance":2}]

[0054] S3. Query the corresponding pre-built knowledge base based on the user's requirement keywords and convert the user's requirement keywords into executable query statements.

[0055] In this embodiment, the internal knowledge base is first queried. The retrieval algorithm is used to find the embedding slices similar to the user's requirements in the knowledge vector database, and the most relevant embedding slices are selected according to the similarity score. Then, the natural language to SQL (or big data query statements such as Spark SQL) conversion technology is used to convert the user's requirements into executable query statements.

[0056] Among them, the conversion from natural language to SQL includes the following steps:

[0057] 1) Word segmentation: Use statistical or rule-based methods, such as regular expressions or conditional random field (CRF) models, to decompose the natural query statement into independent words or phrases.

[0058] 2) Stop word removal: Through a predefined stop word list, remove common words that have no actual contribution to the query results, such as "de", "shi", etc., thereby effectively improving the query efficiency.

[0059] 3) Part-of-speech tagging: Use advanced algorithms such as hidden Markov model (HMM) or conditional random field (CRF) to accurately tag the part of speech of each word (such as noun, verb, etc.), laying a solid foundation for subsequent processing.

[0060] 4) Synonym replacement: By querying the internal knowledge base, replace the queried synonyms with unified words to ensure the accuracy and consistency of the query.

[0061] 5) Data source verification: Based on the internal knowledge base, identify whether the data belongs to a common database or a big data platform to ensure that the data source is valid and matches the user's query requirements;

[0062] 6) Refine query conditions: Refine query conditions by analyzing the context information of natural query statements, keyword logical relationships, and user query intent;

[0063] 7) Evaluation of candidate basic fields: Use TF-IDF, BM25 or a deep learning-based ranking model to evaluate the relevance, importance and query result quality of each candidate basic field to the query intent, and retain highly relevant basic fields.

[0064] Since natural language description may have limitations on user intent, in this embodiment, by importing an existing research report, etc., the analysis logic of the research report can be reversed and a workflow can be generated.

[0065] The specific implementation steps include:

[0066] 1) determining whether a preset indicator word exists in the natural language question;

[0067] 2) Determine whether there are similar reports in the uploaded research report, and determine whether there is a target existing report in the multiple existing reports based on the similarity of each existing report in the multiple existing reports with respect to the natural language question.

[0068] 3) Determine whether there is a completed analysis workflow for the report under the target condition; if it exists, reuse the workflow analysis logic; if it does not exist or there is a report with partially similar logic, proceed to the next step.

[0069] 4) Understand the structure of a research report: A research report includes an introduction, literature review, research methods, data analysis and results presentation, discussion and analysis, and conclusion. Use a macro model to confirm the overall framework and logic of the report.

[0070] 5) Identify core ideas and arguments: The research report will present core ideas. You need to identify these ideas and the arguments that support them, and understand how the author constructs logic through data and analysis.

[0071] 6) Analytical Data and Methods: Review the data and analytical methods used in the research report. Find out the data sources, data processing methods, and analytical tools to build the basis of the report logic.

[0072] 7) Sort out the logical framework: Sort out the logical framework based on the content of the report. For example, if it is an industry analysis, the supply and demand relationship model may be used more; if it is a company analysis, it is necessary to analyze the main business, market size and competition pattern.

[0073] 8) Workflow implementation: Determine input and output: clarify the input (such as data source) and output (such as analysis results) of each node task; use appropriate model methods to process data and perform analysis based on the analysis logic.

[0074] 9) Testing and verification: Verify the correctness of the workflow logic by testing each node, and optimize the nodes through customized settings to improve efficiency.

[0075] 10) Review and optimization: The results need to be reviewed to check whether they are consistent with the conclusions of the original research report, and the workflow needs to be further optimized to improve accuracy and efficiency.

[0076] S4, construct prompt words based on user demand keywords and query statements, input them into the large language model for processing, and convert them into workflow steps to be executed;

[0077] This embodiment uses user needs, query statements, constructs prompt words, and related data as parameters, and submits them to the large language model (LLM) for processing after combination. LLM uses its powerful language understanding, context perception and generation capabilities to convert input into executable workflow steps. Its main process is task decomposition: decompose the tasks in user needs into multiple subtasks, each of which can be further refined into more specific steps. According to the results of task decomposition, select the corresponding node type to build the node. For example: load the workflow configuration information related to data query into the data node so that it can be called in the workflow.

[0078] In this embodiment, the prompt words are constructed as follows:

[0079] #Role

[0080] You are an experienced software architect who is good at translating user requirements into clear and efficient workflows.

[0081] #Good at

[0082] - Understand user needs and translate them into workflows.

[0083] -Be familiar with various types of workflow nodes and understand their functions and usage.

[0084] -Master Python and JavaScript programming languages ​​and be able to write code to achieve specific functions.

[0085] -Familiar with database operations and able to use database nodes to read and write data.

[0086] -Understand the application of large language models and be able to write prompt words suitable for large model nodes to better generate content.

[0087] -Familiar with nodes such as text processing and intent recognition.

[0088] #follow

[0089] -Workflow design should be concise, efficient and easy to understand.

[0090] -The function of each node should be clear and unambiguous.

[0091] -The connections between nodes should be logically clear to avoid dead loops or logical errors.

[0092] -Code should be standardized and easy to read.

[0093] -Try to use existing tools and APIs to avoid reinventing the wheel.

[0094] # Available node types

[0095] -Plug-in nodes (access real-time data and perform external operations by adding tools, including external APIs supported by the system, such as: steel spot data query interface)

[0096] -Large model node (calls the large language model and uses variables and prompt words to generate responses)

[0097] -Code node (can write in python and javascript)

[0098] -Scientific computing nodes (precisely process data with the help of scientific mathematical formulas)

[0099] -Knowledge base node (in the selected knowledge, recall the best matching information according to the input variables and return it in list form)

[0100] -Selector node (connect multiple downstream branches, if the set conditions are met, only the corresponding branch will be run, if none of them are met, only the "otherwise" branch will be run)

[0101] - Loop nodes (used to repeatedly execute a series of tasks by setting the number of loops and logic)

[0102] -Intent recognition node (used to recognize the intent of user input and match it with preset intent options.)

[0103] -Text processing node (used to process the format of multiple string type variables)

[0104] -Message node (supports message output in the intermediate process, supporting both streaming and non-streaming modes)

[0105] - Question and answer node (supports asking questions to users in the middle, supports both preset options and open-ended questions)

[0106] -Variable nodes (Used to read and write variables in the robot. The variable names must match the variable names in the robot.)

[0107] -Database node (can support the release of read and write control on the Database, and users can read and write the internal relational database of the enterprise)

[0108] -Big data nodes (can support querying of big data systems, and users can read and write big data stored within the enterprise)

[0109] -Long-term memory node (used to call long-term memory and obtain the user's personalized information)

[0110] # Output example

[0111] ###Number: Node 1

[0112] Node type: Database node

[0113] Node name: Query Shanghai market steel prices

[0114] Input variable input: query requirements

[0115] Output variable output: query sql

[0116] Configuration config:

[0117] -Prompt: "You are an experienced programmer, you are good at using SQL statements to query data {{Query SQL}}"

[0118] In this embodiment, when the user inputs "analyze the price trend of Shanghai steel and predict the next three months", the large model analysis steps are broken down as follows:

[0119] 1. Determine the data source. If there is a corresponding industry database locally, it will be automatically configured. If not, the corresponding API data can be called;

[0120] 2. Obtain historical data on Shanghai steel prices from selected data sources (at least covering the most recent period of time);

[0121] 3. Clean the acquired data and remove invalid data (such as wrong data, duplicate data, etc.);

[0122] 4. Use appropriate data analysis tools (such as Excel, Python data analysis library, etc.) to organize and visualize the cleaned data (for example, draw a price trend line chart) to analyze the recent steel price trend;

[0123] 5. Select appropriate forecasting models (such as time series analysis models, etc.) based on historical data;

[0124] 6. Use the selected forecasting model and combine relevant factors of the current market (such as policies, supply and demand, etc.) to forecast the price trend of Shanghai steel in the next three months;

[0125] 7. Summarize the results of analysis and prediction in a clear and understandable way (e.g., write a report, make a presentation, etc.)

[0126] In the output example, the node configuration output (taking node 5 as an example) is as follows:

[0127] Node 5, price prediction.

[0128] ###Number: Node 5

[0129] Node type: Large model node

[0130] Node Name: Select Prediction Model

[0131] Input variable input: cleaned steel price data

[0132] Output variable output: prediction model name

[0133] Configuration config:

[0134] -Prompt: "You are a data analysis expert. Based on the steel price data provided, please recommend a suitable forecasting model (such as a time series analysis model, etc.)"

[0135] S5, converting the workflow steps to be executed into workflow node information; converting the workflow node information into a standard business process model;

[0136] This embodiment automatically converts each node information configuration into a standard business process model that is recognizable by a workflow engine and complies with BPMN or other standard process modeling symbols, including graphic elements and symbols.

[0137] The requirements need to be converted into a structured data format and a specific data structure (such as JSON, XML, etc.). For example, create an XML file in a BPMN (Business Process Model and Notation) standardized format, created according to step decomposition and node configuration. Each node and line has corresponding 'id' and 'name' attributes, as well as necessary configuration information. 'bpmndi:BPMNDiagram' defines the layout and node positions of the flowchart. The file is verified by the file validation system to determine whether it complies with the standard specifications. If it does not comply, further node repair is performed to ensure that the file can be loaded correctly. This XML file is then imported into a process design tool that supports BPMN to visualize and further edit the workflow. The execution order and conditions of the business process are defined through configuration files (supporting XML, JSON, YAML and other formats).

[0138] S6. Load the standard business process model into the workflow engine and execute it, returning the execution result.

[0139] In this embodiment, the XML file supporting BPMN or other standard specification files are loaded into the supported workflow process design tool. After the workflow is configured, the user can trigger the execution of the workflow. The workflow engine parses the rule file at startup, registers the components, and assembles them into executable business processes. When executing the process, each business node will be executed in the order of the rule configuration, and the context data will be managed to ensure the correct execution of the process.

[0140] During the execution process, if there are new user requirements, the workflow node information will dynamically adjust the node configuration until the execution ends;

[0141] In this embodiment, the user can integrate the new node into the workflow through simple configuration. According to the needs of data analysis, the user can continue to input natural language requirements, further configure the dynamic nodes, and automatically add, configure, and delete nodes.

[0142] The dynamic adjustment of node configuration includes:

[0143] 1) Automatically add or delete processing nodes, such as large model nodes, plug-in nodes, workflow nodes, image flow nodes, etc., according to the needs of data analysis and the output results of large models;

[0144] 2) Support dynamic adjustment of node parameters or logic in the workflow.

[0145] 3) Based on the analysis of the process by the big model, make suggestions for improvement, such as simplifying steps, merging tasks, and further optimizing the work.

[0146] Further explained:

[0147] Natural language parsing: By describing requirements in natural language, such as "add a data verification node after data cleaning", the system can accurately understand and parse these instructions.

[0148] Automatic node addition: Automatically add new nodes at the corresponding positions of the workflow based on the user's natural language requirements, and configure the necessary parameters and properties.

[0149] Flexible configuration adjustment: Users can adjust the configuration of nodes in the workflow at any time, such as modifying the processing logic of the node, adjusting the input and output of the node, etc., to meet the ever-changing data analysis needs.

[0150] Node deletion and reorganization: Supports node deletion and reorganization functions, removes unnecessary nodes as needed, or rearranges the order of nodes to optimize the execution efficiency of the workflow.

[0151] Finally, the execution results are sent to the user terminal in the form of graphics or reports.

[0152] In this embodiment, the final result is automatically and accurately sent to the user terminal in the form of diversified graphics or refined reports according to the actual situation of the workflow and the user's preset preferences.

[0153] Among them, include:

[0154] Graphical display: Supports intuitive presentation of complex execution results in graphical form, such as flowcharts, pie charts, bar charts, etc. These graphics are not only beautiful and easy to read, but also can clearly display the execution path of the workflow, the status of key nodes and the flow of data, helping users to quickly grasp the overall execution overview.

[0155] Refined report generation: For scenarios that require in-depth analysis, detailed reports can be automatically generated. These reports contain key indicators, data statistics, abnormal situations and treatment measures during the execution process, providing users with comprehensive data support to facilitate subsequent business decisions or optimization adjustments.

[0156] Personalized customization: Taking into full consideration the personalized needs of users, users are allowed to customize the form and content of result display according to their own needs. Whether it is the type, color, layout of the graph, or the fields, sorting, and filtering conditions of the report, users can flexibly configure them to meet the viewing needs in different scenarios.

[0157] Smart push and notification: Once the workflow is completed, the results will be automatically sent to the user's designated terminal, such as enterprise WeChat, WeChat, SMS, APP notification, email, etc. At the same time, it also supports setting reminders to ensure that users can obtain execution results as soon as possible and respond to business changes in a timely manner.

[0158] Data security and privacy protection: In the process of displaying and sending results, we always follow strict data security and privacy protection principles. By adopting advanced technical means such as encrypted transmission and access control, we ensure the security, integrity and confidentiality of the execution results, so that users can enjoy the service without worries.

[0159] The method of the present invention is mainly used in scenarios such as big data analysis and intelligent decision support. By introducing a large model drive, efficient, flexible and intelligent data analysis can be achieved.

[0160] Using advanced large language model technology, business workflow tasks are automatically constructed based on the natural language description provided by the user. It also has the ability of automatic self-correction and automatic execution, which significantly improves the convenience and efficiency of business process construction. It not only enables non-professional users to easily solve practical business problems through simple language descriptions, but also effectively solves many difficult problems in the current technical field.

[0161] This invention cleverly makes up for the lack of intelligence in large models by introducing a workflow mechanism. Through clear workflow task arrangement, we improve the availability and reliability of large models in complex scenarios, thus providing users with a more stable and efficient experience.

[0162] According to step S6, this embodiment loads the standard XML file converted in step S5, or other widely recognized standard specification files, into the workflow process design tool.

[0163] After successfully loading the file, users can use the intuitive interface of the workflow design tool to conveniently configure the various links and parameters of the workflow. Once the configuration is completed, users only need to trigger it to start the execution of the workflow. In this process, the system's built-in workflow engine will play a core role.

[0164] When the workflow engine is started, it will first parse the loaded rule file to ensure that the logic and order of each business node are accurate. Then, the engine will automatically register the required components and intelligently assemble these components into executable business processes according to the rule file. This automated process not only improves work efficiency, but also significantly reduces the risk of human error.

[0165] During the execution of the process, the workflow engine will strictly follow the order of the rule configuration and execute each business node in sequence. At the same time, the engine is also responsible for managing contextual data to ensure that relevant data can be accurately and efficiently transmitted and updated during the process execution. This mechanism effectively guarantees the correct execution of the process and avoids business interruptions or errors caused by inconsistent or lost data.

[0166] In addition, during the data analysis process, nodes in the workflow can select large model nodes, etc., and use the powerful capabilities of large models to infer and predict data. This includes feature extraction nodes and machine learning model nodes, which can deeply explore the potential value of data and provide strong support for decision-making. When subsequent nodes are executed, the system can directly read the output data of the previous node and use it for calculation and analysis of the current node, thereby ensuring the continuity of the workflow and seamless data transfer. At the same time, the system also supports multiple execution methods, such as running the current node, running to the current node, running from the current node, and running all, etc., to meet the data processing needs in different scenarios.

[0167] In addition, in order to address the timeliness limitation of large model training data, the present invention adopts the function calling method to ensure that the model can obtain the latest and most accurate data. At the same time, when faced with the situation where it is difficult to achieve the desired effect using prompt words, the present invention achieves accurate processing of complex tasks by constructing structured workflow instructions and optimizing them through repeated iterations. For example, in the scenario of data analysis report writing, the workflow can orderly guide the model to complete steps such as data collection, outline generation, chapter division, content writing and summarization, thereby producing a well-structured and content-rich analysis elevation.

[0168] In this embodiment, the large model can be based on one or more pre-trained language models, such as chatgpt, Tongyi Qianwen, Doubao, Wenxin Yiyan, etc., and supports function calling. Function calling gives the large language model stronger autonomy and the ability to connect with the outside world. After the model recognizes the user's intention, it selects the most suitable function and fills in the method parameters to return data. Based on the information in the returned tool_calls node, the corresponding method is called locally.

[0169] Define accurate and clear prompts and functions, including detailed and unambiguous descriptions, reasonable parameter settings, etc. To handle invalid JSON or fictitious parameters that may be returned by the model, corresponding fault tolerance and processing mechanisms are required in the local code. Ensure that the model accurately recognizes user intent and selects the correct function to call, which requires high requirements for understanding user input and model training.

[0170] To parse and process the returned data structure, it is necessary to accurately extract the information in the tool_calls node and convert it into locally executable operations.

[0171] Reference Figure 2 The function call schematic diagram in the workflow shown is as follows:

[0172] Step S201: Obtaining user natural language input

[0173] Step S202: Intent recognition, constructing prompt words, and function declaration

[0174] Step S203: Pass the provided prompt words and function declarations (Function Declaration(s)) into the workflow node and call the large model

[0175] Step S204: Based on the function declaration, the model will give tool selection suggestions and related request parameters. Note that the model will only output the suggested tools and request parameters, and will not actually call the function.

[0176] Step S205: The workflow node calls the relevant API according to the model response

[0177] Step S206: re-enter the response content of the API and pass it to the next node. If it is the end node, generate human-readable content. If it is not the end node, go back to step 3 and pass the response content to the next node. Perform corresponding operations according to the type of the next node.

[0178] Step S207: Return the final response to the user, and the user can continue to ask questions.

[0179] In the whole process, the system uses the Function Calling model to complete the syntactic analysis and intent recognition of keywords, define prompt words and function declarations, and finally call the subsequent processing function, that is, the workflow engine generates the function. In this process, Function Calling will pass the necessary parameters (such as the converted SQL statement) to the subsequent function to ensure the smooth progress of the entire processing flow.

[0180] For example:

[0181] {"choices":[{"index":0,"message":{"content":"","role":"assistant","tool_calls":[{"function":{"arguments":"{\"raw_sql\":\"SELECT*FROM pricesWHEREˋtypeˋ='Shanghai steel prices'ANDˋdateˋ>=DATE_SUB(CURDATE(),INTERVAL1MONTH)\"}","name":"func_nl2sql"}}]},"stop_reason":""}],"model":"FunctionCalling"}

[0182] The above JSON example shows how the Function Calling model receives SQL statements converted from natural query statements and passes them as parameters to a processing function named func_nl2sql (natural language to sql) to further perform data query and processing tasks.

[0183] Embodiment 2

[0184] The embodiment of the present invention discloses a workflow engine data analysis system based on a large model drive, referring to Figure 3 As shown, it includes the following modules:

[0185] User input module: used to obtain natural language input requirements from users;

[0186] Preprocessing module: extract keywords based on natural language requirements and sort them by importance to obtain sorted user requirement keywords;

[0187] Large language model processing module: used to query the corresponding preset knowledge base based on user demand keywords and convert user demand keywords into query statements to be executed;

[0188] It is also used to construct prompt words based on user demand keywords and query statements, and input them into the large language model for processing and conversion into workflow steps to be executed;

[0189] Workflow construction module: used to convert the workflow steps to be executed into workflow node information; convert the workflow node information into a standard business process model;

[0190] Tools and plug-in modules: used to load the standard business process model into the workflow engine and execute it, and return the execution results.

[0191] The system of the present invention can use deep learning models to intelligently identify the magnitude and type of data, and can accurately distinguish between small-scale structured data and large-scale, diverse unstructured data. According to the source and magnitude of the data, the system will intelligently choose to use SQL, Spark SQL or other database query languages ​​for querying, and use caching mechanisms such as Redis to improve the reading speed of commonly used data. At the same time, the system will also configure corresponding data processing resources for each task node in the workflow, such as computing resources, storage resources, data interfaces, etc., to ensure the efficient execution of query and computing tasks.

[0192] For small-scale data, the system will connect it to a common relational database and intelligently generate SQL statements based on the user's natural language questions to perform data query and calculation tasks. At the same time, it uses cache technology to improve data reading speed and interacts with the database through a unified data bridge component to achieve seamless data transmission and sharing.

[0193] For large-scale data, the system will connect it to the big data platform and intelligently generate Spark SQL statements based on the user's natural language questions to execute data queries and computing tasks. The workflow engine will create a dedicated path on each node to save intermediate data results, and store this data in the distributed file storage system HDFS to achieve persistent storage and seamless transmission of data. At the same time, the system will also configure specific task execution logic for each node, including script calls, API calls, data input and output, etc., to meet the data processing requirements of different nodes.

[0194] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0195] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A workflow engine data analysis method based on a large model drive, characterized in that: The following steps are involved: S1. Obtain natural language requirements input by users; S2. Based on the natural language demand, extract keywords and sort them by importance to obtain sorted user demand keywords; S3, querying a corresponding preset knowledge base based on the user demand keyword, and converting the user demand keyword into a query statement to be executed; S4, constructing prompt words according to the user demand keywords and query statements, and inputting them into the large language model for processing, and converting them into workflow steps to be executed; S5, converting the workflow steps to be executed into workflow node information; converting the workflow node information into a standard business process model; S6. Load the standard business process model into the workflow engine and execute it, and return the execution result.

2. A workflow engine data analysis method based on large model drive as claimed in claim 1, characterized in that: Step S3, based on the user demand keyword, query the corresponding preset knowledge base and convert the user demand keyword into a query statement to be executed; specifically includes: Decomposing the natural query sentence of the user demand keyword into independent words; Remove some words that have no actual contribution to the query results; Perform part-of-speech tagging on the remaining words; and replace synonyms in the remaining words with unified words; Determine the corresponding preset knowledge base, analyze the context information, keyword logical relationship and user query intention of the natural query statement, refine the query conditions, and generate the corresponding query statement to be executed; The relevance, importance and query result quality of each candidate basic field to the user's query intent are evaluated, and highly relevant basic fields are retained.

3. A workflow engine data analysis method based on large model drive as claimed in claim 2, characterized in that: The step S3 further includes: when the user intention cannot be determined based on the user demand keyword, importing a research report similar to the user demand keyword; The step S4 also includes: generating workflow steps to be executed according to the reverse analysis logic of the research report.

4. A workflow engine data analysis method based on large model drive as claimed in claim 3, characterized in that: Based on the research report, reverse analysis logic is used to generate workflow steps to be executed, including: Determine whether there is a research report similar to the user demand keyword in the research report, and obtain a target report; If there is a completed analysis workflow in the target report, the logical framework of the workflow is reused; If there is no completed analysis workflow in the target report, analyze the structure of the report, obtain the overall framework and logic of the report, identify the core ideas and arguments, and analyze the data and analysis methods used in the report to sort out the logical framework; The logical framework is used to generate workflow steps to be executed.

5. A workflow engine data analysis method based on large model drive as claimed in claim 4, characterized in that: According to the research report, the analysis logic is reversed to generate the workflow steps to be executed; and also includes: Verify the correctness of the workflow logic framework by testing the nodes in the workflow steps, and customize the settings through the nodes to improve efficiency; Review the workflow steps to check whether they are consistent with the conclusions of the original research report, and further optimize the workflow steps to improve accuracy and efficiency.

6. A workflow engine data analysis method based on large model drive as claimed in claim 2, characterized in that: In step S4, the constructing of prompt words specifically includes: Use ICIO or CO-STAR as the prompt word framework; Set different prompt word templates for different scenarios to organize and express needs; Provide specific background information for data analysis and expected answer direction, and determine the content of the prompt words.

7. A workflow engine data analysis method based on large model drive as claimed in claim 6, characterized in that: In step S4, the input large language model is processed and converted into workflow steps to be executed, specifically including: Decomposing the tasks in the constructed prompt words and user requirements into multiple subtasks; Each subtask is further broken down into more specific steps; According to the result of task decomposition, select the corresponding node type to build the node.

8. A workflow engine data analysis method based on large model drive as claimed in claim 7, characterized in that: In step S5, the workflow steps to be executed are converted into workflow node information; the workflow node information is converted into a standard business process model; specifically including: Setting dependencies and execution order between nodes according to the workflow steps; The obtained node information configuration is automatically converted into a standard business process model that follows BPMN or other standard process modeling symbols and can be recognized by the workflow engine.

9. A workflow engine data analysis method based on large model drive as claimed in claim 8, characterized in that: The step S6 further includes: During the execution process, if there are new user requirements, the workflow node information will dynamically adjust the node configuration; Automatically add or delete processing nodes and dynamically adjust node parameters or logic according to the needs of data analysis and the output results of large language models; Based on the analysis of the execution process by the large language model, the execution steps are simplified and the execution tasks are merged.

10. A workflow engine data analysis system based on large model drive, characterized in that: Includes the following modules: User input module: used to obtain natural language input requirements from users; Preprocessing module: extract keywords based on the natural language demand and sort them by importance to obtain sorted user demand keywords; Large language model processing module: used to query the corresponding preset knowledge base based on the user demand keywords, and convert the user demand keywords into query statements to be executed; It is also used to construct prompt words according to the user's demand keywords and query statements, and input them into the large language model for processing and convert them into workflow steps to be executed; Workflow construction module: used to convert the workflow steps to be executed into workflow node information; convert the workflow node information into a standard business process model; Tools and plug-in modules: used to load the standard business process model into the workflow engine and execute it, and return the execution result.

Images