Machine vision solution automatic generation method based on large language model

Abstract

The invention provides a machine vision solution automatic generation method based on a large language model. The module comprises a workflow agent, a parameter agent, an algorithm agent, a workflow engine and a pre-implementation operator library. According to the method, many problems in a traditional workflow-based machine vision solution construction process are solved to a great extent, including that workflow construction highly depends on personal experience of engineers, algorithm parameter performance is sensitive and difficult to adjust, and an application iteration debugging period is long. Besides, the method provided by the invention has multiple flexible deployment modes, the method can be deployed in carriers such as a client field industrial personal computer, a client field private cloud server, a public cloud server and the like, and a machine vision solution automatic generation task is completed. The solution deployment efficiency of a machine vision provider is greatly improved, and the personnel cost of the machine vision provider is reduced; the method is of great significance in improving the efficiency of the last kilometer of a machine vision industry chain.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision applications, and in particular relates to a method for automatically generating a machine vision solution based on a large language model. Background Art

[0002] Machine vision is a rapidly developing branch of artificial intelligence that aims to use machines to replace human eyes for tasks such as measurement, judgment, and recognition. The core of this technology is to convert the target into an image signal through an image acquisition device (such as a CCD or CMOS camera), and then transmit it to a dedicated image processing system, which converts it into a digital signal by analyzing pixel distribution, brightness, color and other information. The image processing system will perform various operations to extract the characteristics of the target, and ultimately make decisions or control the actions of field equipment based on these characteristics.

[0003] The mature large language model technology in recent years has brought new development opportunities and challenges to machine vision software:

[0004] The main users of machine vision software at present, machine vision solution providers, often need to go deep into the customer site and consume a lot of manpower, material resources and time costs to build and adjust the parameters of machine vision solutions: this problem has not been improved at all even with the high use of the machine vision software mentioned by the above manufacturers. Machine vision engineers still need to use machine vision software to build workflows that meet the needs of customer sites and adjust and debug them repeatedly to determine the links and parameters.

[0005] After the workflow of the machine vision software is deployed to the customer's site, the information hidden behind it is very professional. Customers do not pay attention to the details of the algorithm and are often helpless when faced with machine vision software. They can only contact the solution provider for after-sales resolution, which is relatively inefficient and increases the customer's cost of using machine vision solutions. Summary of the invention

[0006] Therefore, based on the above problems, the present invention uses the latest large language model technology to provide a machine vision solution automatic generation technology. On the one hand, it solves the problem of machine vision solution providers consuming too high deployment cycles and personnel costs at customer sites; on the other hand, it reduces the cost of using machine vision solutions at customer sites. Only by describing the on-site problems in human language can the intelligent agent be called to modify the current workflow parameters or even rearrange the workflow. It has great practical significance for improving the efficiency of the "last mile" of machine vision deployment.

[0007] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0008] A system for automatically generating machine vision solutions based on a large language model, comprising:

[0009] The multi-agent module is used to understand the input human language text according to the preheating prompt words and output the workflow representation and operator implementation code;

[0010] The pre-implemented operator library is used to store manually written pre-implemented operators and operator implementation codes output by the multi-agent module, and select operators from them to form workflows;

[0011] The workflow engine is used to parse the workflow representation output by the multi-agent module, load the operators in the pre-implemented operator library, run the workflow, and when it receives the machine vision application task trigger signal, parse the signal and trigger the corresponding workflow to complete the real-time reasoning of the machine vision application task.

[0012] The multi-agent module comprises:

[0013] Workflow agent, which is used to understand the input human language text according to the preheating prompt words, combine the operators in the pre-implemented operator library based on their specifications and functions, generate workflows, and output workflow representations that meet user needs and queues of operators that lack intermediate elements;

[0014] A parameter agent, which is used to understand and fine-tune the input workflow representation according to the warm-up prompt words, and output a workflow representation containing a strategy that meets the parameter adjustment required by the user;

[0015] The operator agent is used to understand the basic information definition and explanation description text of the missing operators in the input missing operator queue according to the preheating prompt words, define the description and structure of the missing operators, and generate and output the operator implementation code.

[0016] The pre-implemented operator library includes 2D machine vision, 3D machine vision, deep learning, machine learning, reinforcement learning, large language model agent, mathematics, and tool-type operator calls, among which:

[0017] 2D machine vision operators include image preprocessing, connected region analysis, result post-processing, and measurement submodules;

[0018] 3D machine vision operators include point cloud information processing and coordinate transformation submodules;

[0019] Deep learning operators include image classification, object detection, instance segmentation, semantic segmentation, posture estimation, and motion prediction submodules;

[0020] Machine learning operators include linear regression, random forest, and support vector machine submodules;

[0021] Reinforcement learning operators include Q-learning, policy-based methods, and Actor-Critic submodules;

[0022] Large language model agent operators include RAG, autogen, and conversational large model submodules;

[0023] Mathematical operators include matrix operations and digital definition submodules;

[0024] Tool operators include image acquisition, signal sending, timing, result conversion, and input and output sub-modules.

[0025] The workflow engine includes: application SDK, REST API service, and machine vision application software package.

[0026] The workflow representation is a structured text representation of a workflow, which is used to accurately describe the workflow using text, and includes the following elements or objects:

[0027] Global timeout: This is a required field and is used to describe the overall timeout for algorithm execution.

[0028] Task List: This is a required field and is used to describe specific tasks.

[0029] The objects in the task list array must contain the following fields:

[0030] Task name: This is a required field and is used to describe the task name. In a workflow, the task name cannot be repeated and is guaranteed by using a dash plus a number to increase automatically.

[0031] Operator dependency: This is a required field and can be an empty array. It is used to describe other tasks that this task depends on. Fill in the task names of other tasks.

[0032] Operator Name: This is a required field and is used to describe the operator name for the specific execution of the task. In this session and the current generation algorithm, the task name is unique, but there can be multiple operator names as needed.

[0033] Operator configuration item: It is an optional item used to describe the configuration of the task and inject the configuration item of the operator;

[0034] Is it asynchronous? It is optional and defaults to No. It is used to describe whether it is an asynchronous task.

[0035] Operator execution condition: It is an optional item used to describe the conditions for operator execution. The conditions are given in the form of key-value pairs and support five types of condition judgments: >=, <=, >, <, =. The condition value supports four types: int, double, float, and string.

[0036] Operator timeout: It is optional and is used to describe the operator timeout.

[0037] The missing operator queue is used to describe the operators and their descriptions that are still missing in the pre-implemented operator library in order to complete the task specified by the current user, and includes the following elements or objects:

[0038] Operator name: used to describe the unique identifier of the operator in the pre-implemented operator library. The name of the operator reflects its basic function.

[0039] Configuration items: An operator can have multiple configuration items. Each configuration item has a name, type, and initial value. The actual value of the configuration item is given an initial value by the workflow agent in the workflow representation and adjusted by the parameter agent. In the missing operator queue, the workflow agent is required to give the name, type, and initial value of the configuration item of the operator to be added, and add a human language text explanation for each configuration item.

[0040] Input items: An operator can have multiple input items. Each input item has an integer index that increases from 0, a type, and a default value. The actual value of the input item cannot be given by any agent, but is calculated in real time by the workflow engine based on the data of upstream and downstream operators during reasoning. In the absence of an operator queue, the workflow agent is required to give the index, type, and default value of the operator input item to be added, and add a human language text explanation for each input item;

[0041] Output item: An operator can only have one output item, which has a type attribute. The actual value of the output item is calculated in real time by the workflow engine based on the data of upstream and downstream operators during reasoning. If there is a lack of operator queue, the workflow agent is required to give the type attribute of the new operator output item and add a human language text explanation.

[0042] The operator implementation code includes:

[0043] Operator declaration: declare the operator with the given operator name, input items, output items, and textual explanations of each item;

[0044] Input declaration: a code snippet that declares and instantiates each input item in an operator from an upstream operator;

[0045] Configuration item declaration: a code snippet that declares and instantiates each configuration item in the operator from the workflow representation;

[0046] Result calculation: The code segment that calculates the result based on the actual values ​​of the input items and configuration items;

[0047] Output item declaration and return: Return the result obtained by the result calculation code segment.

[0048] A method for automatically generating a machine vision solution based on a large language model comprises the following steps:

[0049] The multi-agent module understands the input human language text according to the preheating prompt words, and outputs the workflow representation and operator implementation code;

[0050] The pre-implemented operator library stores manually written pre-implemented operators and operator implementation codes output by the multi-agent module, and selects operators from them to form a workflow;

[0051] The workflow engine parses the workflow representation output by the multi-agent module, loads the operators in the pre-implemented operator library, and runs the workflow. When it receives a machine vision application task trigger signal, it parses the signal and triggers the corresponding workflow to complete the real-time reasoning of the machine vision application task.

[0052] The multi-agent module performs the following steps:

[0053] The workflow agent understands the input human language text according to the preheating prompt words, combines the operators in the pre-implemented operator library based on their specifications and functions, generates a workflow, and outputs a workflow representation that meets the user's needs and a queue of operators that lack intermediate elements;

[0054] The parameter agent understands and fine-tunes the input workflow representation based on the warm-up prompt words, and outputs a workflow representation containing a strategy that meets the parameter adjustment required by the user;

[0055] The operator agent understands the basic information definition and explanatory description text of the missing operators in the input missing operator queue according to the preheating prompt words, defines the description and structure of the missing operators, and generates and outputs the operator implementation code.

[0056] The present invention has the following beneficial effects and advantages:

[0057] 1. The present invention effectively shortens the average deployment cycle and personnel costs consumed by machine vision solution providers at customer sites.

[0058] 2. The present invention effectively reduces the cost of using machine vision solutions for customers.

[0059] 3. The present invention effectively improves the application efficiency of machine vision solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0061] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0062] In view of the problems existing in the prior art in the above analysis, the present invention provides a complete set of automatic generation methods for machine vision solutions based on large language models. The algorithm includes Figure 1 Modules and elements shown.

[0063] like Figure 1 ,The algorithm is divided into five modules: [Workflow Agent], [Parameter Agent], [Operator Agent], [Pre-implemented Operator Library], [Workflow Engine] and three elements: workflow representation, missing operator queue and operator implementation code. The technical details of these five modules and three elements will be elaborated in detail below.

[0064] Module Introduction

[0065] Module 1: Workflow Agent

[0066] The workflow agent is specifically characterized by: preheating prompt words, input link, core model, and output link.

[0067] The warm-up prompt words are a piece of human language text pre-input into the core model conversation; the purpose is to let the workflow agent understand its own role and the correct format of its output link in advance; at the same time, let the core model understand the specifications and functions of the operators already in the pre-implemented operator library, and tend to let the core model give priority to the operators therein to combine workflows.

[0068] The input link is the human language text that is input into the core model in real time, and is also the user's input text in the entire method; its purpose is to let the workflow agent understand the specific details of the machine vision application process required by the user.

[0069] The core model, as the core component of the workflow agent, can be any conversational agent on the market; its purpose is to understand the human language in the input link and output a workflow representation that meets the user's needs.

[0070] The output link is the real-time output of the core model. Its purpose is to output the machine vision application generated by the intelligent agent based on its own understanding in the form of an intermediate element workflow representation so that it can be loaded and processed in subsequent steps. At the same time, the output of the intermediate element lacks the operator queue.

[0071] Module 2 Parameter Agent

[0072] The parameter agent is specifically characterized by: preheating prompt words, input link, core model, and output link.

[0073] The warm-up prompt words are a piece of human language text pre-input into the core model session; the purpose is to let the parameter intelligent agent understand its own role and the correct format of its output link in advance.

[0074] The input link is the representation text input into the core model in real time, and the workflow representation output by the workflow agent is used as input.

[0075] The core model, as the core component of the workflow agent, can be any conversational agent on the market. Its purpose is to understand the workflow representation of the input link, output a strategy that meets the parameter adjustments required by the user, and fine-tune the input workflow representation.

[0076] The output link is the real-time output of the core model. Its purpose is to output the machine vision application generated by the intelligent agent based on its own understanding in the form of a workflow representation so that it can be loaded and processed in subsequent steps.

[0077] Module 3 Operator Agent

[0078] The algorithm agent is specifically characterized by: preheating prompt words, input link, core model, and output link.

[0079] The warm-up prompt words are a piece of human language text pre-input into the core model conversation; the purpose is to let the algorithm agent understand its own role and the correct format of its output in advance.

[0080] The input link is used as a representation text input into the core model in real time. The basic information definition and explanation description text of the missing operators in the missing operator queue generated in the output link of the workflow agent in module 1 are input into the core model.

[0081] The core model, as the core component of the workflow agent, can be any conversational agent on the market. Its purpose is to understand the description and structural definition of the missing operators in the missing operator queue of the input link; and to generate operator implementations on the target programming language code based on this information.

[0082] The output link is the real-time output of the core model. Its purpose is to update the missing operator implementations generated by the intelligent agent based on its own understanding to the pre-implemented operator library of module 4 for subsequent calls.

[0083] Module 4 Pre-implemented Operator Library

[0084] In order to make the workflow agent in module 1 understand the task more efficiently, the present invention sets a pre-implemented operator library manually written based on industry experience, and tends to let it select operators from the pre-implemented operator library to form a workflow. The pre-implemented operator library for machine vision tasks includes but is not limited to operator calls of types such as 2D machine vision, 3D machine vision, deep learning, machine learning, reinforcement learning, large language model agents, mathematics, and tools. 2D machine vision operators include modules such as image preprocessing, connected region analysis, result postprocessing, and measurement; 3D machine vision operators include submodules such as point cloud information processing and coordinate transformation; deep learning operators include submodules such as image classification, target detection, instance segmentation, semantic segmentation, posture estimation, and action prediction; machine learning operators include submodules such as linear regression, random forest, and support vector machine; reinforcement learning operators include submodules such as Q learning, policy-based methods, and Actor-Critic; large language model intelligent agents include submodules such as RAG, autogen, and conversational large models; mathematical operators include submodules such as matrix operations and digital definitions; tool operators include submodules such as image acquisition, signal transmission, timing, result conversion, and input and output. By pre-implementing the operator library itself, the needs of most machine vision solutions on the market can be basically covered. It is worth noting that there are still some machine vision tasks that require users to customize operators on this basis to meet special needs, which is also the core meaning and value of the present invention.

[0085] The operators are all implemented in high-level programming languages ​​and can be directly loaded and executed by the workflow engine. In addition, the pre-implemented operator library is updated in real time, and new operators can be added to it by humans or the output link of the module 3 operator agent, as the implemented operators of the module 1 workflow agent pre-prompt word link.

[0086] Module 5: Workflow Engine

[0087] The workflow engine is the core component responsible for parsing the workflow representation of the final output of the parameter agent, loading the operators in the pre-implemented operator library, and finally running the workflow. The workflow engine should be able to be flexibly released in various forms according to user needs, including but not limited to: application SDK, REST API service, machine vision application software package, etc., and deployed in the customer's production environment. When the machine vision application task trigger signal arrives, the signal is parsed and the corresponding workflow is triggered to complete the real-time reasoning of the machine vision application task.

[0088] Introduction of elements

[0089] Element 1: Workflow Representation

[0090] Workflow representation is a structured text representation of a workflow. It can be implemented based on structured text representation formats such as JSON, XML, YAML, and HTML. Its purpose is to accurately describe the workflow using text so that the agent can output the actual tasks it understands in the form of a workflow.

[0091] A workflow generally contains multiple task nodes, each of which is an instance of an operator in the workflow. The operator provides the specific implementation of the task node, including input, output, configuration items, and implementation code. It can be directly loaded and called by the workflow engine.

[0092] In order to more accurately describe the structure of a workflow and the properties of its operators, a workflow representation must include the following elements or objects:

[0093] Global timeout: Required. The default value is 5000 milliseconds, which is the overall timeout for algorithm execution.

[0094] Task list: Required. Add all task descriptions internally. The objects in the array must contain the following fields:

[0095] Task name: required. Task names cannot be repeated in a workflow. Use a hyphen plus a number to ensure that the name is auto-incremental.

[0096] Operator dependency: required, can be an empty array. Task dependency, array. Fill in the names of other tasks that this task depends on.

[0097] Operator Name: Required. This is the name of the operator that the task executes. In this session and the current generation algorithm, the task name is unique, but there can be multiple operator names as needed.

[0098] Operator configuration items: optional. Task configuration, configuration items injected into the operator.

[0099] Asynchronous or not: optional, default is no. Whether it is an asynchronous task. Notify module 5 workflow engine to execute the operator asynchronously. That is, it will not wait for the operator to return the result, and call the downstream operator immediately after publishing the task.

[0100] Operator execution condition: optional. The operator execution condition is given in the form of key-value pairs. Supports: >=, <=, >, <, = five types of condition judgments, and the condition value supports four types: int, double, float, string.

[0101] Operator timeout: optional, default is 500 milliseconds, the operator timeout.

[0102] Element 2: Lack of operator queue

[0103] The missing operator queue is one of the outputs generated by the workflow agent in module 1. It is used to describe the operators and their descriptions that are still missing in the pre-implemented operator library in order to complete the task specified by the current user. The list formed by these operators and descriptions is the missing operator queue.

[0104] The description of an operator must include at least the following elements or objects:

[0105] Operator name: The unique identifier of the operator in the pre-implemented operator library. The name of the operator should generally reflect its basic function.

[0106] Configuration items: An operator can have multiple configuration items, each with a name, type, and initial value. The actual values ​​of these configuration items can be given initial values ​​by the workflow agent in the workflow representation and adjusted by the parameter agent. In the missing operator queue, the workflow agent is required to give the name, type, and initial value of the configuration item of the operator to be added; and add a human language text explanation for each configuration item.

[0107] Input: An operator can have multiple inputs, each with an integer index starting from 0, a type, and a default value. Unlike configuration items, the actual value of an input cannot be given by any agent, but is calculated in real time by the workflow engine based on the data of upstream and downstream operators during reasoning. In the missing operator queue, the workflow agent is required to give the index, type, and default value of the operator input that needs to be added. A human language text explanation is added for each input.

[0108] Output item: An operator can only have one output item, and the output item has a type attribute. Similar to the input item, the value of the output item is calculated in real time by the workflow engine based on the data of the upstream and downstream operators during reasoning. In the missing operator queue, the workflow agent is required to give the type attribute of the operator output item that needs to be added. And add a human language text explanation.

[0109] Element 3 operator implementation code

[0110] The operator implementation code is the output generated by the operator agent and is a high-level language code that can be directly parsed and loaded by the workflow engine. It must include the following parts:

[0111] Operator declaration: declare the operator according to the given operator name, input items, output items, text explanations and other information.

[0112] Input declaration: A code snippet that declares and instantiates each input item in an operator from an upstream operator.

[0113] Configuration item declaration: A code snippet that declares and instantiates each configuration item in an operator from a workflow representation.

[0114] Result calculation: The code segment that calculates the result based on the actual values ​​of the input items and configuration items.

[0115] Output item declaration and return: Return the result obtained by the result calculation code segment.

[0116] Introduction to module implementation method

[0117] Module 1-3 Workflow Agent, Parameter Agent, Operator Agent

[0118] All intelligent agents use the "Wenxin Yiyan" product launched by Baidu.

[0119] Module 4 Pre-implemented Operator Library

[0120] In order to make as many operators as possible achieve higher operating efficiency, the manually written pre-implemented operators are written in C / C++. Including operator types and some other types of operators. The scale of the pre-implemented operator library is as high as 152 operators.

[0121] Module 5: Workflow Engine

[0122] The workflow engine responsible for concatenating operators is implemented in Python to achieve better scalability of workflow representation and support diversity of operator forms.

[0123] 2.2 Introduction to the implementation methods of the elements

[0124] Element 1-2 Workflow representation, lack of operator queue

[0125] The workflow is represented using HTML structured text language, mainly considering the convenience of front-end rendering during application. The lack of operator queues is described in simplified Chinese.

[0126] Element 3 operator implementation code

[0127] The operator agent uses Python to write the implementation code of the missing operators to minimize the difficulty of understanding the core model.

Claims

1. A system for automatically generating machine vision solutions based on a large language model, characterized in that: include: The multi-agent module is used to understand the input human language text according to the preheating prompt words and output the workflow representation and operator implementation code; The pre-implemented operator library is used to store manually written pre-implemented operators and operator implementation codes output by the multi-agent module, and select operators from them to form workflows; The workflow engine is used to parse the workflow representation output by the multi-agent module, load the operators in the pre-implemented operator library, run the workflow, and when it receives the machine vision application task trigger signal, parse the signal and trigger the corresponding workflow to complete the real-time reasoning of the machine vision application task.

2. The automatic generation system of machine vision solutions based on a large language model according to claim 1, characterized in that: The multi-agent module comprises: Workflow agent, which is used to understand the input human language text according to the preheating prompt words, combine the operators in the pre-implemented operator library based on their specifications and functions, generate workflows, and output workflow representations that meet user needs and queues of operators that lack intermediate elements; A parameter agent, which is used to understand and fine-tune the input workflow representation according to the warm-up prompt words, and output a workflow representation containing a strategy that meets the parameter adjustment required by the user; The operator agent is used to understand the basic information definition and explanation description text of the missing operators in the input missing operator queue according to the preheating prompt words, define the description and structure of the missing operators, and generate and output the operator implementation code.

3. The automatic generation system of machine vision solutions based on a large language model according to claim 1, characterized in that: The pre-implemented operator library includes 2D machine vision, 3D machine vision, deep learning, machine learning, reinforcement learning, large language model agent, mathematics, and tool-type operator calls, among which: 2D machine vision operators include image preprocessing, connected region analysis, result post-processing, and measurement submodules; 3D machine vision operators include point cloud information processing and coordinate transformation submodules; Deep learning operators include image classification, object detection, instance segmentation, semantic segmentation, posture estimation, and motion prediction submodules; Machine learning operators include linear regression, random forest, and support vector machine submodules; Reinforcement learning operators include Q-learning, policy-based methods, and Actor-Critic submodules; Large language model agent operators include RAG, autogen, and conversational large model submodules; Mathematical operators include matrix operations and digital definition submodules; Tool operators include image acquisition, signal sending, timing, result conversion, and input and output sub-modules.

4. The automatic generation system of machine vision solutions based on a large language model according to claim 1, characterized in that: The workflow engine includes: application SDK, REST API service, and machine vision application software package.

5. The automatic generation system of machine vision solutions based on a large language model according to claim 2, characterized in that: The workflow representation is a structured text representation of a workflow, which is used to accurately describe the workflow using text, and includes the following elements or objects: Global timeout: This is a required field and is used to describe the overall timeout for algorithm execution. Task List: This is a required field and is used to describe specific tasks.

6. The automatic generation system of machine vision solutions based on a large language model according to claim 5, characterized in that: The objects in the task list array must contain the following fields: Task name: This is a required field and is used to describe the task name. In a workflow, the task name cannot be repeated and is guaranteed by using a dash plus a number to increase automatically. Operator dependency: This is a required field and can be an empty array. It is used to describe other tasks that this task depends on. Fill in the task names of other tasks. Operator Name: This is a required field and is used to describe the operator name for the specific execution of the task. In this session and the current generation algorithm, the task name is unique, but there can be multiple operator names as needed. Operator configuration item: It is an optional item used to describe the configuration of the task and inject the configuration item of the operator; Is it asynchronous? It is optional and defaults to No. It is used to describe whether it is an asynchronous task. Operator execution condition: It is an optional item used to describe the conditions for operator execution. The conditions are given in the form of key-value pairs and support five types of condition judgments: >=, <=, >, <, =. The condition value supports four types: int, double, float, and string. Operator timeout: It is optional and is used to describe the operator timeout.

7. The automatic generation system of machine vision solutions based on a large language model according to claim 2, characterized in that: The missing operator queue is used to describe the operators and their descriptions that are still missing in the pre-implemented operator library in order to complete the task specified by the current user, and includes the following elements or objects: Operator name: used to describe the unique identifier of the operator in the pre-implemented operator library. The name of the operator reflects its basic function. Configuration items: An operator can have multiple configuration items. Each configuration item has a name, type, and initial value. The actual value of the configuration item is given an initial value by the workflow agent in the workflow representation and adjusted by the parameter agent. In the missing operator queue, the workflow agent is required to give the name, type, and initial value of the configuration item of the operator to be added, and add a human language text explanation for each configuration item. Input items: An operator can have multiple input items. Each input item has an integer index that increases from 0, a type, and a default value. The actual value of the input item cannot be given by any agent, but is calculated in real time by the workflow engine based on the data of upstream and downstream operators during reasoning. In the absence of an operator queue, the workflow agent is required to give the index, type, and default value of the operator input item to be added, and add a human language text explanation for each input item; Output item: An operator can only have one output item, which has a type attribute. The actual value of the output item is calculated in real time by the workflow engine based on the data of upstream and downstream operators during reasoning. If there is a lack of operator queue, the workflow agent is required to give the type attribute of the new operator output item and add a human language text explanation.

8. The automatic generation system of machine vision solutions based on a large language model according to claim 2, characterized in that: The operator implementation code includes: Operator declaration: declare the operator with the given operator name, input items, output items, and textual explanations of each item; Input declaration: a code snippet that declares and instantiates each input item in an operator from an upstream operator; Configuration item declaration: a code snippet that declares and instantiates each configuration item in an operator from the workflow representation; Result calculation: The code segment that calculates the result based on the actual values ​​of the input items and configuration items; Output item declaration and return: Return the result obtained by the result calculation code segment.

9. A method for automatically generating a machine vision solution based on a large language model, characterized in that: The following steps are involved: The multi-agent module understands the input human language text according to the preheating prompt words, and outputs the workflow representation and operator implementation code; The pre-implemented operator library stores manually written pre-implemented operators and operator implementation codes output by the multi-agent module, and selects operators from them to form a workflow; The workflow engine parses the workflow representation output by the multi-agent module, loads the operators in the pre-implemented operator library, and runs the workflow. When it receives a machine vision application task trigger signal, it parses the signal and triggers the corresponding workflow to complete the real-time reasoning of the machine vision application task.

10. The method for automatically generating a machine vision solution based on a large language model according to claim 9, characterized in that: The multi-agent module performs the following steps: The workflow agent understands the input human language text according to the preheating prompt words, combines the operators in the pre-implemented operator library based on their specifications and functions, generates a workflow, and outputs a workflow representation that meets the user's needs and a queue of operators that lack intermediate elements; The parameter agent understands and fine-tunes the input workflow representation based on the warm-up prompt words, and outputs a workflow representation containing a strategy that meets the parameter adjustment required by the user; The operator agent understands the basic information definition and explanatory description text of the missing operators in the input missing operator queue according to the preheating prompt words, defines the description and structure of the missing operators, and generates and outputs the operator implementation code.

Images