Question and answer mode scientific data processing workflow arrangement method and device based on large language model

Abstract

The invention discloses a question and answer mode scientific data processing workflow arrangement method and device based on a large language model, and belongs to the technical field of computers. The method comprises the following steps: constructing an operator and a workflow knowledge base; performing workflow retrieval in the operator and workflow knowledge base based on the natural language description of the target task, and after embedding k workflow expressions obtained by the natural language description and retrieval into a prompt template, obtaining a task job extension sequence based on a large model; performing operator retrieval in an operator and workflow knowledge base based on the task job extension sequence, and after embedding the natural language description, the task job extension sequence and n operators obtained by retrieval into a prompt template, obtaining a task job reorganization sequence based on a large model; and converting the task job reorganization sequence into a workflow in a workflow description language form. According to the method, the complete scientific data workflow is automatically generated through the natural language, and the use threshold of the workflow is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a method and device for orchestrating a question-and-answer-based scientific data processing workflow based on a large language model. Background Art

[0002] Scientific data refers to the original data and its derivative data generated during scientific research and experiments. The generation and processing of this data often involve multiple steps and stages, featuring a process-oriented nature. Scientific data workflows are of crucial significance in the entire scientific research and data processing process. It constructs an orderly framework for data processing and analysis, making complex data processing processes manageable. Through workflows, scattered tools and algorithms for different data processing links can be integrated together to ensure the accuracy and coherence of data throughout the processing process.

[0003] Currently popular technologies available for workflow orchestration mainly fall into two categories. One is the method of forming a directed acyclic graph (DAG) by dragging and dropping on a visual interface. This method is intuitive for simple workflows, but as the complexity of the workflow increases, the visual interface may become very crowded, increasing the operation difficulty. The other is text-based configuration through a certain workflow description language (such as WDL, CWL, XML, JSON). Although this method is more suitable for describing complex workflows, it requires users to have certain programming and text editing capabilities. However, in practical applications, domain scientists face many difficulties when using traditional scientific workflow software. Domain scientists have rich professional knowledge in their own research fields, but they often lack the relevant skills for operating workflow software, especially for those workflow generation methods based on dragging and dropping on a visual interface or complex text description languages. This skill gap leads to low efficiency or even inability to complete when they transform their research ideas into actual executable workflows.

[0004] Large language models (LLMs) are deep learning models trained based on massive text data, showing excellent capabilities in the field of natural language processing. It can not only generate natural language text but also deeply understand the meaning of the text and process various natural language tasks, such as text summarization, question answering, translation, etc. Especially in understanding complex instructions and generating accurate context-aware code, large models have unique advantages. It can receive requirements in the form of natural language and attempt to generate corresponding code or logical structures to meet these requirements. For example, when given a natural language description of a data analysis task, the large model can attempt to generate a preliminary workflow architecture.

[0005] However, large models also face problems such as hallucinations and time misalignment. Since the datasets used for training large models, although huge in scale, still cannot cover all knowledge domains, situations inconsistent with the real world will occur in the generated text, namely the hallucination phenomenon. Such hallucinations may manifest as answering off-topic, providing incorrect information, etc., affecting the reliability of large models. In addition, the time misalignment problem may also cause large models to generate content based on outdated knowledge.

[0006] Existing research has shown that combining external knowledge with internal knowledge can effectively alleviate hallucinations, especially for knowledge-intensive tasks such as the scientific data workflow orchestration field involved in the present invention. During the long-term development process, current workflow software has accumulated a large number of operators and workflows. As the basic functional units in workflows, operators represent various data processing operations, such as data reading, calculation, storage, etc. There are complex dependencies between these operators, and this dependency determines the execution order and logic of the workflow. This rich operator and workflow information can be used as an external knowledge base to provide strong guidance for large models to generate workflows, enabling large models to utilize these existing knowledge structures to better complete the workflow orchestration task.

[0007] In addition, existing workflow description languages have some limitations. These workflow description languages often consume a large number of tokens when representing complex workflows, which not only increases the processing cost but may also lead to inefficiencies or even exceed the processing capacity during the processing of large models. Summary of the Invention

[0008] To address the above problems, the present invention proposes a question-and-answer-based scientific data processing workflow orchestration method and device based on large language models. This method is targeted at domain scientists and, by leveraging the context understanding ability, language organization ability, and semantic expansion of large models, supports users to automatically generate complete scientific data workflows in a natural language manner, reducing the usage threshold of workflows.

[0009] To achieve the above object, the technical solution of the present invention includes the following content.

[0010] A question-and-answer-based scientific data processing workflow orchestration method based on large language models, the method comprising:

[0011] Construct an operator and workflow knowledge base;

[0012] Based on the natural language description D of the target task, perform workflow retrieval in the operator and workflow knowledge base, and after embedding the natural language description D and the k retrieved workflow expressions into the task job extension sequence generation prompt template, obtain the task job extension sequence based on the large model;

[0013] Retrieve operators in the operator and workflow knowledge base based on the task operation extended sequence, and after embedding the natural language description D, the task operation extended sequence, and the retrieved n operators into the task operation reorganization sequence to generate a prompt template, obtain the task operation reorganization sequence based on the large model;

[0014] Convert the task operation reorganization sequence into a workflow in the form of a workflow description language.

[0015] Further, construct an operator and workflow knowledge base, including:

[0016] Collect operator documents and workflow documents;

[0017] Split the operator documents and workflow documents based on operator records and workflows to obtain operators and workflows with semantic independence;

[0018] Generate a workflow representation for each workflow; wherein, the workflow representation includes: job number, operator name, and input port configuration information of the job, and the input port configuration information of the job is used to describe which upstream job each input port of the job is docked with;

[0019] Encode each operator and workflow into a vector representation;

[0020] Generate an operator and workflow knowledge base based on operators, operator vector representations, workflow representations, and workflow vector representations.

[0021] Further, the operator record includes: operator name, description, input parameters, and input and output ports.

[0022] Further, retrieving operators in the operator and workflow knowledge base based on the task operation extended sequence includes:

[0023] Encode each step of the task operation extended sequence to generate a vector representation v;

[0024] Match the vector representation v with the operator vector representation to obtain n operators.

[0025] Further, the specification formats of the task operation extended sequence and the task operation reorganization sequence are consistent with the specification format of the workflow representation, and each step in the task operation extended sequence and the task operation reorganization sequence only contains one operator.

[0026] Further, the form of the workflow description language includes: WDL, CWL, XML, or JSON.

[0027] Further, after converting the task operation reorganization sequence into a workflow in the form of a workflow description language, it further includes:

[0028] Get user's evaluation of the output workflow;

[0029] It is determined whether to add the workflow to the operator and workflow knowledge base according to the evaluation.

[0030] A question-answering scientific data processing workflow arrangement device based on a large language model, the device comprising:

[0031] Knowledge base module, used to build operator and workflow knowledge base;

[0032] A task job extraction extension module is used to perform workflow retrieval in the operator and workflow knowledge base based on the natural language description D of the target task, and embed the natural language description D and the k retrieved workflow expressions into the task job extension sequence generation prompt template, and then obtain the task job extension sequence based on the large model;

[0033] A task operation sequence reorganization module performs operator retrieval in the operator and workflow knowledge base based on the task operation extension sequence, and embeds the natural language description D, the task operation extension sequence and the retrieved n operators into a task operation reorganization sequence generation prompt template, and obtains the task operation reorganization sequence based on the large model;

[0034] The scientific data workflow generation module converts the task job reorganization sequence into a workflow in the form of a workflow description language.

[0035] An electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method for arranging a workflow of scientific data processing based on a large language model in a question-and-answer style is implemented.

[0036] A computer-readable storage medium, characterized in that computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, any of the above-mentioned question-and-answer scientific data processing workflow orchestration methods based on a large language model is implemented.

[0037] Compared with the prior art, the present invention has at least the following beneficial effects.

[0038] (1) The present invention supports the automatic generation of scientific data workflows by leveraging the context understanding and language organization capabilities of large models, thereby lowering the threshold for using workflows.

[0039] (2) Expansion and optimization of workflow orchestration process: Combine the large model and professional workflow knowledge base to expand the description information of the workflow in the question, further improve the accuracy of local knowledge base search and the executability of the generated workflow.

[0040] (3) Structural optimization of the workflow representation model: Provide a workflow representation model for large models, which can achieve efficient transmission and accurate understanding of workflow organization information while consuming fewer tokens. Description of the Drawings

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments.

[0042] Figure 1 It is a schematic flowchart of a method for choreographing a question-and-answer-based scientific data processing workflow based on a large language model provided by the present invention.

[0043] Figure 2 It is an example operator in the operator document of the present invention.

[0044] Figure 3 It is an example workflow of the workflow document of the present invention.

[0045] Figure 4 It is an example task job sequence extended by the present invention.

[0046] Figure 5 It is a reorganized job sequence of the example task of the present invention.

[0047] Figure 6 It is an executable JSON workflow of the example task of the present invention.

[0048] Figure 7 It is a schematic diagram of the execution of the JSON workflow of the example task of the present invention.

[0049] Figure 8 It is a schematic structural diagram of a device for choreographing a question-and-answer-based scientific data processing workflow based on a large language model provided by the present invention. Specific implementation method

[0051] To make the above solutions and beneficial effects of the present invention more obvious and understandable, the following will be described in detail through embodiments and in conjunction with the drawings.

[0052] The present invention provides a method for choreographing a question-and-answer-based scientific data processing workflow based on a large language model, as Figure 1 shown, including the following steps:

[0053] 1. Preparation of the operator and workflow knowledge base.

[0054] Although it is possible to directly receive a simple natural language description of a task from domain scientists and attempt to directly generate a workflow, it may lead to low-quality workflow generation and subsequent workflow execution failures. Therefore, it is necessary to pre-build an operator and workflow knowledge base to provide a basis for enriching and expanding tasks. The knowledge base only needs to be built once and then updated as needed.

[0055] The process of preparing the knowledge base mainly includes the following three steps:

[0056] 1) Use a document loader to load operator and workflow documents.

[0057] The operator document contains basic information such as the operator name, description, input parameters, input and output ports in structured JSON or CSV format.

[0058] The workflow document contains the collected existing workflow organization information, including operators and the dependencies between operators.

[0059] In order to enable the large model to well understand the workflow organization information while consuming fewer tokens, the present invention proposes a workflow representation model for the large model, mainly in the form of: id|name|inport 1 <<id up1 ; inport 2 <<id up2 .

[0060] Where id is the job number, name is the job name (operator name), inport is the input port configuration information of the job, and "|" is the delimiter for each column. Among them, the "job input port configuration information" describes which upstream job each input port of the job is docked to. Each job can have multiple input ports, such as "inport 1 ", "inport 2 "; "<< " represents the docking behavior, and "id up1 " "id up2 " represents a certain upstream job to which it is docked. The delimiter between multiple input port configuration information is "; "; the "job number" is a hierarchical and ordered number, and the job number starts from level 2, that is, 1.1, 1.2,... Job 1.2 is the upstream of job 1.1 (when a certain input port of job 1.2 is docked to job 1.1). In order to ensure the independence of each workflow in the workflow document, each level 1 number (i.e., 1, 2,...) represents a workflow, and the name column corresponding to the level 1 number is the workflow name.

[0061] 2) Split the knowledge base using a text splitter. Although the operators and workflow documents of the present invention are structured, proper splitting is still required to adapt to the context processing ability of the LLM. Since structured documents (such as JSON or CSV) have clear field and record delimiters, the present invention utilizes these characteristics for intelligent splitting. For operator documents, they can be split by operator records, and each operator record contains information such as its name, description, input parameters, input and output ports, etc. For workflow documents, they can be split by workflow, ensuring that each split block can completely express a workflow organization information while maintaining its semantic independence from other blocks. Such a splitting method can avoid the "semantic relevance" problem and effectively utilize the context window of the LLM.

[0062] 3) Use a text embedder to encode each block into a floating-point vector. These vectors can capture the semantic features of the text, making similar texts closer in the vector space. Next, the present invention stores these vector representations in a vector database as the basis for subsequent question-and-answer workflow orchestration. Each vector only needs to be stored once instead of generating a new embedding every time it is used. During the orchestration process, when the user describes a task in natural language, the present invention can retrieve the operator and workflow vectors related to the task from the knowledge base and embed them into the context of the LLM.

[0063] 2. Extraction and Expansion of Task Jobs

[0064] The goal of this step is to extract a structured task job sequence from the simple natural language description of the task by domain scientists. To improve the completeness of the task job sequence, the present invention proposes a task job sequence expansion method as shown in the following formula:

[0065] Q = {P 1 (D, {F 1 、F 2 、...F k}), L}

[0066] where Q is the task job expansion sequence, P 1 is the prompt containing the natural language description D and the relevant historical workflow F, and L is the large language model used. Using the natural language description as the retrieval condition, k relevant historical workflows in the knowledge base are retrieved, where k can be customized, and they are embedded into the context of the large model. Then, by comprehensively utilizing the language understanding ability, endogenous knowledge, and external workflow knowledge of the large model, the task job sequence is expanded. In addition, P 1 needs to specify the canonical format for the large model to output the task job sequence and ensure that each step in the sequence is an atomic operation, that is, it only contains one operator. The canonical format is as follows:

[0067] ```

[0068] csv

[0069] id|name

[0070] ```

[0071] Among them, the id is a hierarchical ordered number, and a 2-level number is sufficient. The name is the job name.

[0072] 3. Reorganization of the task job sequence

[0073] This step is crucial for ensuring the normal execution of the final workflow. Due to the divergence of the large model, the operators included in the generated task sequence may not exist, so it is impossible to ensure that the extracted task job sequence fully meets the user's intention and can run normally. Therefore, it is necessary to reorganize the task job sequence generated in the second stage, as shown in the following formula:

[0074] RQ = {P 2 (D, Q, F, {O 1 、O 2 、...On}), L}

[0075] Among them, RQ is the reorganized task job sequence, and P 2 is the prompt word that includes the natural language description D, the task job extension sequence Q, a certain relevant historical workflow F, and the relevant operator O. L is the large language model used. First, each step of the task job sequence generated in the second stage is encoded into a vector representation form, and then vector similarity retrieval is performed one by one to retrieve n relevant operators, and n can be customized. In addition, after removing duplicates from all the retrieved operators, they are embedded into the context of the large model. Using the context understanding ability of the large model, the most suitable operator is selected for each step of the sequence, and the dependency relationship between the operators is determined, so as to finally generate an executable workflow. In addition, custom operator extension is supported. If the operator is not appropriate, scientists can add custom operators by uploading operators. Similarly, P 2 needs to specify the standard format for the large model to output the task job reorganization sequence. The standard format is consistent with the workflow representation model of the present invention, but the id is a 2-level number, and it is not necessary to include the workflow number and the workflow name.

[0076] 4. Generation of scientific data workflow

[0077] Convert the task job reorganization sequence generated in stage 3 into a workflow description language that can be executed on the computing framework, such as WDL, CWL, XML, JSON, etc.

[0078] 5. Establish a user feedback mechanism, store the generated workflow representation model that satisfies users in the workflow knowledge base, and enrich the workflow knowledge base.

[0079] Next, the present invention will be specifically described by taking the "production of windbreak and sand fixation function data products" workflow from the scientific data collaborative analysis platform BigFlow as an example.

[0080] 1) Preparation of operators and workflow knowledge base. Prepare operator documents. For example, as Figure 2 shown, the operator document contains basic information such as the operator name, description, input parameters, input and output ports in structured JSON or CSV format.

[0081] Prepare workflow documents. For example, as Figure 3 shown, convert the existing workflow organization information into the workflow representation model for large models provided by the present invention, which includes operators and the dependency relationships between operators.

[0082] Among them, the first-level number 1 is the workflow splitting identifier, and the name after the first-level number is displayed as the workflow name. 1.1 to 1.6 are task job sequences, and each step in the sequence is an atomic operation, that is, it only contains one operator. Jobs 1.1 to 1.4 have no upstream jobs, that is, the input ports do not receive any data. Job 1.5 receives four upstream data. Among them, the windSpeedPort input port receives the output data of Job 1.1, the snowCapPort input port receives the output data of Job 1.2, the soilMoisturePort input port receives the output data of Job 1.3, and the vegetationCoveragePort input port receives the output data of Job 1.4. The downstream job of Job 1.5 is Job 1.6, and Job 1.6 has only one default input port Default.

[0083] After splitting the knowledge base using a text splitter, use an embedding model (such as multilingual-e5-large) to map the text blocks into corresponding vector representations and save them in the vector library for later vector retrieval.

[0084] 2) Extraction and expansion of task jobs. Taking the question "Please provide the production process of windbreak and sand fixation data products for ecological stations" as an example, after receiving a simple description of the task in this step, first use the natural language description as the retrieval condition to retrieve k relevant historical workflows in the knowledge base, support customizing k (in this example k = 1), and embed them into the context of the large model. Then, comprehensively utilize the language understanding ability, endogenous knowledge, and external workflow knowledge of the large model to expand the task job sequence. The finally generated task job sequence is as Figure 4 shown.

[0085] 3) Reorganization of the task operation sequence. Based on the task operation sequence generated in step 2), relevant operators in the operator knowledge base are retrieved. According to the input and output of the operators and the reference of the historical workflow, the task operation sequence is reorganized, and a final workflow representation model is generated, such as Figure 5 shown.

[0086] 4) Generation of the scientific data workflow. The workflow representation model is converted into a workflow description language that can be executed on the computing framework, such as WDL, CWL, XML, JSON, etc. In this example, it is converted into the JSON format required by BigFlow, such as Figure 6 shown. The JSON workflow is rendered to BigFlow and successfully run, such as Figure 7 shown.

[0087] 5) User feedback. Through the user feedback entry, satisfaction information is collected, and the workflow representation model that the user is satisfied with is stored in the workflow knowledge base to enrich the workflow knowledge base.

[0088] Based on the same concept, the present invention also provides a question-and-answer-based scientific data processing workflow orchestration device 800 based on a large language model, such as Figure 8 shown, including: a knowledge base module 801, a task operation extraction and extension module 802, a task operation sequence reorganization module 803, a scientific data workflow generation module 804, and a user feedback module 805. Among them, the knowledge base module 801 is used to build, update, and retrieve operator and workflow knowledge bases; the task operation extraction and extension module 802 is used to extract and extend the structured task operation sequence from the simple natural language description of the task; the task operation sequence reorganization module 803 is used to select the most suitable operator for each step of the sequence and determine the dependency relationship between the operators to generate a complete workflow representation model; the scientific data workflow generation module 804 is used to convert the workflow representation model generated by the task operation sequence reorganization module into a workflow description language that can be executed on the computing framework, such as WDL, CWL, XML, JSON, etc.; the user feedback module 805 is used to establish a user feedback mechanism and store the generated workflow representation model that the user is satisfied with in the workflow knowledge base to enrich the workflow knowledge base.

[0089] Although specific embodiments of the present invention are disclosed for illustrative purposes, the purpose is to help understand the content of the present invention and implement it accordingly. Those skilled in the art can understand that: without departing from the spirit and scope of the present invention and the appended claims, various substitutions, changes, and modifications are possible. Therefore, the present invention should not be limited to the content disclosed in the best embodiments, and the scope of protection required by the present invention is defined by the scope of the claims.

Claims

1. A method for arranging a question-answering scientific data processing workflow based on a large language model, characterized in that: The method comprises: constructing an operator and a workflow knowledge base; Based on the natural language description D of the target task, a workflow search is performed in the operator and workflow knowledge base, and after the natural language description D and the retrieved k workflow expressions are embedded into the task job extension sequence generation prompt template, the task job extension sequence is obtained based on the large model; Performing operator retrieval in the operator and workflow knowledge base based on the task operation extension sequence, and embedding the natural language description D, the task operation extension sequence and the retrieved n operators into a task operation reorganization sequence generation prompt template, and obtaining the task operation reorganization sequence based on the large model; Convert the task job reorganization sequence into a workflow in the form of a workflow description language.

2. The method according to claim 1, characterized in that: Build operator and workflow knowledge base, including: Collect operator documents and workflow documents; Split operator documents and workflow documents based on operator records and workflows to obtain operators and workflows with semantic independence; Generate a workflow description for each workflow; wherein the workflow description includes: a job number, an operator name, and input port configuration information of the job, wherein the input port configuration information of the job is used to describe that each input port of the job is connected to an upstream job; Encode each operator and workflow into a vector representation; Based on the operator, the operator vector representation, the workflow expression and the workflow vector representation, an operator and workflow knowledge base is generated.

3. The method according to claim 2, characterized in that The operator record includes: the name, description, input parameters and input and output ports of the operator.

4. The method according to claim 2, characterized in that Performing operator retrieval in the operator and workflow knowledge base based on the task job extension sequence includes: Encode each step of the task operation expansion sequence to generate a vector representation v; Match the vector representation v with the operator vector representation to obtain n operators.

5. The method according to claim 1, characterized in that: The standard formats of the task operation extension sequence and the task operation reorganization sequence are consistent with the standard format of the workflow representation, and each step in the task operation extension sequence and the task operation reorganization sequence contains only one operator.

6. The method according to claim 1, characterized in that The workflow description language includes: WDL, CWL, XML or JSON.

7. The method according to any one of claims 1 to 6, characterized in that: After converting the task job reorganization sequence into a workflow in the form of a workflow description language, the method further includes: Get user's evaluation of the output workflow; It is determined whether to add the workflow to the operator and workflow knowledge base according to the evaluation.

8. A question-answering scientific data processing workflow arrangement device based on a large language model, characterized in that: The device comprises: a knowledge base module, used for constructing an operator and workflow knowledge base; A task job extraction extension module is used to perform workflow retrieval in the operator and workflow knowledge base based on the natural language description D of the target task, and embed the natural language description D and the k retrieved workflow expressions into the task job extension sequence generation prompt template, and then obtain the task job extension sequence based on the large model; A task operation sequence reorganization module performs operator retrieval in the operator and workflow knowledge base based on the task operation extension sequence, and embeds the natural language description D, the task operation extension sequence and the retrieved n operators into a task operation reorganization sequence generation prompt template, and obtains the task operation reorganization sequence based on the large model; The scientific data workflow generation module converts the task job reorganization sequence into a workflow in the form of a workflow description language.

9. An electronic device, characterized in that: The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method for arranging a workflow of question-answering scientific data processing based on a large language model as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the method for arranging a workflow of question-and-answer scientific data processing based on a large language model as described in any one of claims 1 to 7 is implemented.

Images