Automatic operator arrangement method and system based on large model

Abstract

The invention relates to an operator automatic arrangement method and system based on a large model, and belongs to the technical field of algorithm engineering. According to the method, operators are defined as minimum calculation units, classified according to functions and stored in an operator library; collecting user requirements, operator contents and workflow data for the fine tuning model, performing data cleaning, and integrating into a fine tuning data set; performing fine tuning on the chatGLM-6B model by using the fine tuning data set, so that the chatGLM-6B model can understand and arrange operators; the model analyzes a user demand, extracts key information, selects an operator from an operator warehouse, and arranges a workflow for a user; the arranged workflow is displayed to a user, feedback of the workflow is collected, and iterative optimization is carried out according to needs; and workflow execution is monitored in real time, optimization suggestions are provided for the workflow, and an execution report is generated. According to the invention, through operator warehouse learning, the adaptation problem of the interface and the parameter is automatically processed, and the complexity is reduced, so that the arrangement of the algorithm workflow is more efficient and targeted.

Description

Technical Field

[0001] The present invention belongs to the field of algorithm engineering technology, and in particular relates to an automatic operator arrangement method and system based on a large model. Background Art

[0002] In recent years, with the rapid development of hardware technology and the explosive growth of data, large models, especially large language models such as chatGLM, have demonstrated amazing performance on a variety of natural language processing tasks. These models usually have billions to trillions of parameters, and their model structures are becoming increasingly complex. By learning from massive amounts of data, these models have been able to understand and generate human language, solve complex tasks, and even surpass traditional algorithmic methods.

[0003] At the same time, intelligent automation, a technology that combines artificial intelligence, machine learning and process automation, is being widely used in various industries. This technology aims to improve efficiency, reduce errors, and achieve business flexibility and sustainability by automating business processes. In particular, intelligent automation has shown great potential in the fields of algorithm development, data processing and workflow orchestration.

[0004] Despite this, algorithm incubation platforms still face huge challenges in responding to user needs, selecting and combining operators, and generating algorithm workflows. Traditional methods usually require manual analysis of needs, manual selection of operators, and setting of parameters, which is not only time-consuming but also prone to errors. Therefore, how to improve the efficiency, accuracy, and flexibility of algorithm workflow generation has become a pressing issue. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] The technical problem to be solved by the present invention is how to provide a method and system for automatic operator arrangement based on a large model to solve the problem of improving the generation efficiency of the algorithm workflow.

[0007] (II) Technical solution

[0008] In order to solve the above technical problems, the present invention proposes an automatic operator arrangement method based on a large model, which comprises the following steps:

[0009] Step S1, define an operator as the smallest unit of calculation, classify it by function and store it in an operator library;

[0010] Step S2: Collect user requirements, operator content, and workflow data for fine-tuning the model, perform data cleaning, and integrate them into a fine-tuning data set;

[0011] Step S3: Use the target task dataset to fine-tune the large model so that it can understand and orchestrate operators;

[0012] Step S4: After the big model analyzes the user's needs and extracts key information, it selects appropriate operators from the operator warehouse and arranges workflows for the user;

[0013] Step S5: Display the arranged workflow to the user, collect their feedback, and perform iterative optimization as needed;

[0014] Step S6: Monitor workflow execution in real time, provide optimization suggestions for workflows, generate execution reports, and use feedback and data to continuously fine-tune the large model.

[0015] The present invention also provides an operator automatic arrangement system based on a large model, the system comprising: an input module, a conversion module, a fine-tuning module, an interaction module, an interpretation module, a background management module and an output module;

[0016] The input module comprises:

[0017] Input interface: allows users to select input methods, including text, voice, image, and video;

[0018] File processing: ensuring that the content uploaded by users is in the correct format and setting size and format restrictions;

[0019] Preprocessing tools: convert speech into text, identify text or key information in images or videos;

[0020] The conversion module comprises:

[0021] CLIP adapter: Use CLIP model to parse non-text input;

[0022] Data matching: Match with text data to find the most relevant description;

[0023] The fine-tuning module comprises:

[0024] Monitoring tools: real-time tracking of operator library updates and user input patterns;

[0025] Online learner: collects user feedback and performs small-scale fine-tuning in real time;

[0026] The interaction module includes:

[0027] Workflow display: Display the workflow generated by the model to the user;

[0028] Live conversation interface: A real-time chat window that allows users to interact with the model;

[0029] Feedback processor: adjusts the model based on user feedback;

[0030] The interpretation module comprises:

[0031] Explanation Generator: Generates explanation text for each choice of the model;

[0032] Interface display tool: Display explanations on the interface and provide interactive buttons for users to query;

[0033] The background management module includes:

[0034] Database management: store and manage operator libraries, user input and feedback;

[0035] Performance monitoring: monitor the performance of models and systems;

[0036] Update scheduling: Update the system based on performance data and feedback;

[0037] The output module comprises:

[0038] Results display: The final workflow design is presented to the user;

[0039] Error Reporting: Generates error reports when problems occur and provides resolution suggestions.

[0040] (III) Beneficial effects

[0041] The present invention proposes a method and system for automatic operator arrangement based on a large model. Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. The present invention provides an automatic operator arrangement method and system based on a large model. The large model automatically handles the adaptability problems of interfaces and parameters through operator warehouse learning, reduces the complexity, and thus makes the arrangement of algorithm workflows more efficient and targeted.

[0043] 2. The present invention provides an automatic operator arrangement method and system based on a large model. By only describing the requirements, the large language model can understand and extract key information, realize fast operator selection and workflow arrangement, and greatly reduce the user's learning cost and manual operation.

[0044] 3. The present invention provides an automatic operator orchestration method and system based on a large model. Through real-time feedback and modification from users, the large model can be continuously fine-tuned and optimized to make its understanding of user needs more accurate. Accordingly, the orchestrated workflow will be more in line with user expectations.

[0045] 4. The present invention provides an automatic operator arrangement method and system based on a large model, and also provides a test running and modification environment, so that users can immediately see the execution effect of the workflow and make necessary adjustments, thereby speeding up the launch of the project.

[0046] 5. The present invention provides an automatic operator arrangement method and system based on a large model. Each manual modification and feedback from the user is a learning opportunity for the large model, making it more and more mature in long-term applications and having higher response speed and accuracy for similar needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a step diagram of the method of the present invention;

[0048] Figure 2 It is a schematic diagram of the overall implementation scheme of the present invention;

[0049] Figure 3 Schematic diagram of the system architecture of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0051] In order to overcome the shortcomings of the prior art, the present invention proposes an operator automatic arrangement method and system based on a large model. Through real-time feedback and modification from users, the large model can be continuously fine-tuned and optimized to make its understanding of user needs more accurate. Accordingly, the arranged workflow will be more in line with user expectations, and can automatically parse the user's algorithm requirements, select appropriate operators from the operator warehouse, and intelligently combine these operators to generate a complete algorithm workflow. This method not only greatly improves efficiency, but also ensures the quality and accuracy of the workflow.

[0052] The present invention proposes an automatic operator arrangement method and system based on a large model, including: defining an operator as the smallest unit of calculation, classifying it by function and storing it in an operator library; collecting user requirements, operator content and workflow data for fine-tuning the model, performing data cleaning, and integrating it into a fine-tuning data set; using the fine-tuning data set to fine-tune the chatGLM-6B model so that it can understand and arrange operators; the model parses user requirements, extracts key information, selects operators from the operator warehouse, and arranges workflows for users; displays the arranged workflows to users, collects their feedback, and performs iterative optimization as needed; monitors workflow execution in real time, provides optimization suggestions for workflows, and generates execution reports. The present invention automatically handles the adaptability issues of interfaces and parameters through operator warehouse learning, reduces complexity, and thus makes the arrangement of algorithm workflows more efficient and targeted.

[0053] To achieve the above object, the present invention provides an automatic operator arrangement method based on a large model, comprising:

[0054] Step S1, define an operator as the smallest unit of calculation, classify it by function and store it in an operator library;

[0055] Step S2: Collect user requirements, operator content, and workflow data for fine-tuning the model, perform data cleaning, and integrate them into a fine-tuning data set;

[0056] Step S3: fine-tune the large model using the target task dataset so that it can understand and orchestrate operators; for example, the chatGLM-6B large model;

[0057] Step S4: After the big model analyzes the user's needs and extracts key information, it selects appropriate operators from the operator warehouse and arranges workflows for the user;

[0058] Step S5: Display the arranged workflow to the user, collect their feedback, and perform iterative optimization as needed;

[0059] Step S6: Monitor workflow execution in real time, provide optimization suggestions for workflows, generate execution reports, and use feedback and data to continuously fine-tune the large model.

[0060] Further, step S2 is specifically as follows:

[0061] Step S21: obtaining text from users describing their computing needs through questionnaires, interviews, or online submissions;

[0062] Step S22: extracting the operator content of each operator from the operator library, including: detailed description, code, input and output format, etc.;

[0063] Step S23: Collect complete workflow data, including user requirements, selected operators, and execution results;

[0064] Step S24: Use tools such as regular expressions to remove unnecessary characters or strings from user requirements, operator content, and workflow data;

[0065] Step S25: using a common stop word library, such as NLTK, to remove non-critical words;

[0066] Step S26: Use NLP tools, such as spaCy or NLTK, to unify the vocabulary into its basic form;

[0067] Step S27: Use a data processing library, such as Pandas, to integrate and format the data to form a fine-tuning dataset to meet the needs of model fine-tuning.

[0068] Further, step S3 is specifically as follows:

[0069] Step S31: Determine the batch size, learning rate and number of iterations for fine-tuning;

[0070] Step S32: data input, dividing the integrated fine-tuning data set into a training set and a validation set;

[0071] Step S33: Use the training set to fine-tune the chatGLM-6B large model, and use the validation set to check the performance of the model.

[0072] Further, step S4 is specifically as follows:

[0073] Step S41: using deep learning or NLP technology to parse the text input by the user and identify its core intent and key information;

[0074] Step S42: Use the large model trained in S33 to score the function of each operator and select the most suitable operator for the user's needs;

[0075] Step S43: Organize the selected operators into a complete workflow according to the order and logic of user requirements.

[0076] Furthermore, step S5 is specifically as follows:

[0077] Step S51: providing a graphical interface for the user to display the recommended workflow structure;

[0078] Step S52: Allow the user to modify the workflow or provide feedback on the interface;

[0079] Step S53: Based on user feedback, the large model re-performs operator matching and workflow orchestration.

[0080] Further, step S6 includes:

[0081] Step S61: deploy monitoring tools, such as Prometheus or Grafana, to track the execution of the workflow;

[0082] Step S62: regularly collect workflow execution time, error rate, resource consumption and other data;

[0083] Step S63: Feed the collected performance data back to the large model to further optimize and fine-tune the large model.

[0084] The details are as follows:

[0085] Step S6311: Establish a monitoring system to track updates to the operator library and changes in user input patterns;

[0086] Step S6312: Generate a fine-tuning trigger mechanism to automatically start fine-tuning when a significant change is detected;

[0087] Step S6313: After each interaction between the user and the model, collect and analyze the user's feedback;

[0088] Step S6314: Perform small-scale fine-tuning on the model based on the feedback.

[0089] Furthermore, it also includes interactive iterative feedback from users, as follows:

[0090] Step S6321: Design an intuitive interface so that users can see the workflow generated for them by the big model;

[0091] Step S6322: providing a real-time chat window on the interface to allow the user to interact with the large model;

[0092] Step S6323: When the user asks or provides feedback, the large model should respond immediately;

[0093] Step S6324: For each question or feedback from the user, the big model should provide a reasonable answer or make corresponding adjustments.

[0094] Furthermore, the model interpretability is enhanced, as follows:

[0095] Step S6331: When the model makes a selection for the user, an explanation text should be automatically generated;

[0096] Step S6332: The explanation should clearly describe why the model makes such a choice;

[0097] Step S6333: On the user interface, there should be a "Why?" button next to each model selection;

[0098] Step S6334: When the user clicks, the explanation provided by the model should be displayed immediately.

[0099] The present invention also provides an operator automatic arrangement system based on a large model, which is used for any one of the above-mentioned operator automatic arrangement methods based on a large model, including: an input module, a conversion module, a fine-tuning module, an interaction module, an interpretation module, a background management module and an output module;

[0100] An input module (Input Module), the input module comprises:

[0101] Input interface: allows users to select input method (text, voice, image, video);

[0102] File processing: ensuring that the content uploaded by users is in the correct format and setting size and format restrictions;

[0103] Preprocessing tools: convert speech into text, identify text or key information in images or videos;

[0104] A transformation module, the transformation module comprising:

[0105] CLIP adapter: Use CLIP model to parse non-text input;

[0106] Data matching: Match with text data to find the most relevant description;

[0107] A fine tuning module (Finetuning Module), the fine tuning module comprising:

[0108] Monitoring tools: real-time tracking of operator library updates and user input patterns;

[0109] Online learner: collects user feedback and performs small-scale fine-tuning in real time;

[0110] An interaction module (Interaction Module), the interaction module comprising:

[0111] Workflow display: Display the workflow generated by the model to the user;

[0112] Live conversation interface: A real-time chat window that allows users to interact with the model;

[0113] Feedback processor: adjusts the model based on user feedback;

[0114] An Explainability Module, the Explainability Module comprising:

[0115] Explanation Generator: Generates explanation text for each choice of the model;

[0116] Interface display tool: Display explanations on the interface and provide interactive buttons for users to query;

[0117] Backend Management Module, the backend management module includes:

[0118] Database management: store and manage operator libraries, user input and feedback;

[0119] Performance monitoring: monitor the performance of models and systems;

[0120] Update scheduling: Update the system based on performance data and feedback;

[0121] An output module (Output Module), the output module comprising:

[0122] Results display: The final workflow design is presented to the user;

[0123] Error Reporting: Generates error reports when problems occur and provides resolution suggestions.

[0124] Embodiment 1:

[0125] Operator Introduction

[0126] Definition: An operator is a predefined program or function that specializes in handling a certain type of task.

[0127] Functional classification:

[0128] Data input operator: reads various data sources.

[0129] Data processing operators: perform operations such as sorting and filtering.

[0130] Data conversion operator: converts data format.

[0131] Data calculation operator: performs mathematical or statistical operations.

[0132] Data output operator: output to the specified target.

[0133] like Figure 1 As shown, the present invention is specifically:

[0134] Step S1: define operators as the smallest unit of calculation, classify them by function and store them in the operator library;

[0135] Step S2: Collect user requirements, operator content, and workflow data for fine-tuning the model, perform data cleaning, and integrate them into a fine-tuning dataset;

[0136] Step S3: Fine-tune the chatGLM-6B model using a specialized dataset to enable it to understand and orchestrate operators;

[0137] Step S4: The model parses user requirements, extracts key information, selects appropriate operators from the operator warehouse, and orchestrates workflows for users;

[0138] Step S5: Display the arranged workflow to the user, collect their feedback, and perform iterative optimization as needed;

[0139] Step S6: Monitor workflow execution in real time, provide optimization suggestions for the workflow, generate execution reports, and use feedback and data to continuously fine-tune the model.

[0140] As a specific implementation, step S2 is specifically as follows:

[0141] Step S21: obtaining text from users describing their computing needs through questionnaires, interviews, or online submissions;

[0142] Step S22: extracting detailed description, code and input and output formats of the operator from the operator library;

[0143] Step S23: Collect complete workflow cases, including user requirements, selected operators, and execution results;

[0144] Step S24: using a regular expression tool to remove unnecessary characters or strings;

[0145] Step S25: using a common stop word library to remove non-critical words;

[0146] Step S26: using NLP tools to unify the vocabulary into its basic form;

[0147] Step S27: Integrate and format the data using a data processing library.

[0148] As a specific implementation, step S3 is as follows:

[0149] Step S31: Determine the batch size, learning rate and number of iterations for fine-tuning;

[0150] Step S32: Divide the integrated data into a training set and a validation set;

[0151] Step S33: Fine-tune chatGLM-6B using the training set and check the performance of the model using the validation set.

[0152] As a specific implementation, step S4 is specifically as follows:

[0153] Step S41: using deep learning or NLP technology to parse the text input by the user and identify its core intent and key information;

[0154] Step S42: using the model to score the functions of the operators and select operators according to user needs;

[0155] Step S43: Organize the selected operators into a complete workflow according to the order and logic of user requirements.

[0156] As a specific implementation, step S5 is specifically as follows:

[0157] Step S51: providing a graphical interface for the user to display the recommended workflow structure;

[0158] Step S52: Allow the user to modify the workflow or provide feedback on the interface;

[0159] Step S53: Based on user feedback, the model re-performs operator matching and workflow orchestration.

[0160] As a specific implementation, step S6 includes:

[0161] Step S61: deploy monitoring tools to track the execution of the workflow;

[0162] Step S62: regularly collect workflow execution time, error rate and resource consumption data;

[0163] Step S63: Feed the collected performance data back to the model to optimize and fine-tune the model.

[0164] As a specific implementation method, adaptive fine-tuning is also included, as follows:

[0165] Step S6311: Establish a monitoring system to track updates to the operator library and changes in user input patterns;

[0166] Step S6312: Generate a fine-tuning trigger mechanism to automatically start fine-tuning when a significant change is detected;

[0167] Step S6313: After each interaction between the user and the model, collect and analyze the user's feedback;

[0168] Step S6314: Perform small-scale fine-tuning on the model based on the feedback.

[0169] Furthermore, it also includes interactive iterative feedback from users, as follows:

[0170] Step S6321: Design an intuitive interface so that users can see the workflow generated for them by the model;

[0171] Step S6322: providing a real-time chat window on the interface to allow the user to interact with the model;

[0172] Step S6323: When the user asks or provides feedback, the model should respond immediately;

[0173] Step S6324: For each question or feedback from the user, the model should provide a reasonable response or make corresponding adjustments.

[0174] Furthermore, the model interpretability is enhanced, as follows:

[0175] Step S6331: When the model makes a selection for the user, an explanation text should be automatically generated;

[0176] Step S6332: The explanation should clearly describe why the model makes such a choice;

[0177] Step S6333: On the user interface, there should be a "Why?" button next to each model selection;

[0178] Step S6334: When the user clicks, the explanation provided by the model should be displayed immediately.

[0179] like Figure 2 As shown, the model needs to be fine-tuned first so that it can understand the input user needs, combine the needs with specific operators, and output the operator arrangement scheme, combining operators of different types and purposes into workflows. Before fine-tuning the model, it is necessary to collect and prepare the data used for fine-tuning. The main methods of data collection are: collecting text data describing users' needs, collecting operator content data, collecting arranged workflow data, etc. After the above data collection is completed, the collected data is cleaned by removing special characters and punctuation, removing stop words, stem extraction, word form restoration, etc., and then the user demand data, operator content data and existing workflow data are combined and arranged into the data format required for model fine-tuning. Use this data to perform user demand analysis and operator workflow arrangement fine-tuning on the model chatGLM-6B, so that it has the ability to automatically arrange workflows by operators.

[0180] After the operator automatic arrangement workflow based on the large model starts, the user demand content is input into the fine-tuned chatGLM-6B large model. The model will understand the user's needs and extract the key information of the user's needs in the sentence.

[0181] Then the model will search for operators that correspond to the user's needs in the operator warehouse based on the user's needs. The large model will select and arrange operators based on the acquired operator information and demand information.

[0182] After the workflow is orchestrated, the model will give feedback to the user on the orchestrated workflow suggestions, allowing the user to provide feedback and confirm the workflow. If the user reports that the workflow has obvious problems, does not meet the user's needs and needs to be modified, the large model will reselect and re-orchestrate the operators. It will generate a new workflow based on the user's needs and return it to the user. If the user believes that the workflow does not need to be modified, the workflow will start running.

[0183] The model will monitor the working status of the running workflow and generate workflow optimization suggestion documents and operation result evaluation and analysis reports. These two documents will be used to optimize workflow arrangement and optimization model, and added to the fine-tuning data to further fine-tune and update the model.

[0184] like Figure 3As shown, an operator automatic arrangement system based on a large model is used for any one of the above-mentioned operator automatic arrangement methods based on a large model, including: an input module, a conversion module, a fine-tuning module, an interaction module, an interpretation module, a background management module and an output module;

[0185] An input module (Input Module), the input module comprises:

[0186] Input interface: allows users to select input method (text, voice, image, video);

[0187] File processing: ensuring that the content uploaded by users is in the correct format and setting size and format restrictions;

[0188] Preprocessing tools: convert speech into text, identify text or key information in images or videos;

[0189] A transformation module, the transformation module comprising:

[0190] CLIP adapter: Use CLIP model to parse non-text input;

[0191] Data matching: Match with text data to find the most relevant description;

[0192] A fine tuning module (Finetuning Module), the fine tuning module comprising:

[0193] Monitoring tools: real-time tracking of operator library updates and user input patterns;

[0194] Online learner: collects user feedback and performs small-scale fine-tuning in real time;

[0195] An interaction module (Interaction Module), the interaction module comprising:

[0196] Workflow display: Display the workflow generated by the model to the user;

[0197] Live conversation interface: A real-time chat window that allows users to interact with the model;

[0198] Feedback processor: adjusts the model based on user feedback;

[0199] An Explainability Module, the Explainability Module comprising:

[0200] Explanation Generator: Generates explanation text for each choice of the model;

[0201] Interface display tool: displays explanations on the interface and provides interactive buttons for users to query;

[0202] Backend Management Module, the backend management module includes:

[0203] Database management: store and manage operator libraries, user input and feedback;

[0204] Performance monitoring: monitor the performance of models and systems;

[0205] Update scheduling: Update the system based on performance data and feedback;

[0206] An output module (Output Module), the output module comprising:

[0207] Results display: The final workflow design is presented to the user;

[0208] Error Reporting: Generates error reports when problems occur and provides resolution suggestions.

[0209] Embodiment 2:

[0210] As a specific implementation plan, a large retailer hopes to better utilize its huge sales data to find hidden market opportunities. Operators based on large models are used to automatically orchestrate workflows as follows:

[0211] First, the large retailer classified and organized its own operator library to ensure that it included key operators such as data input, data conversion, time series analysis, geographic data analysis, and data visualization. Then, the retailer provided sales data for the past year, which included information such as sales date, product type, sales region, and sales volume. In order to better enable the model to understand this data, they cleaned it, including removing outliers, unifying naming conventions for different regions, and formatting dates.

[0212] With this data, they fine-tuned the model to make it more suitable for their own business needs. After fine-tuning, users can interact with the model directly. For example, a marketing employee may ask through voice commands: "I want to know the monthly sales trends of various regions last year." This voice request will be converted into text, and then the model will start its automatic arrangement work.

[0213] The model first parses the text, identifies the user's core needs, and then selects appropriate operators from the operator library. This may include data input operators, time series analysis operators, geographic data analysis operators, etc. When the workflow is initially designed, it will be presented to the user, who can review, verify, or make modifications.

[0214] In some cases, the model may also automatically identify new requirements or operator patterns. For example, when the model finds that data in certain regions may have seasonal changes, it may automatically add a time series analysis operator.

[0215] Once the workflow is completed, the model not only displays the results to the user, but also provides explanations for each step of the decision, helping the user understand the logic behind its recommendations. In addition, the model will continue to monitor the execution of the workflow to ensure that all data is processed correctly.

[0216] This method enables retailers to obtain a series of clear and accurate sales data analysis reports without spending a lot of time and resources on manual data processing and analysis. This not only helps them quickly understand product sales trends and market opportunities, but also enables them to make more informed market decisions.

[0217] Compared with the prior art, the present invention has the following beneficial effects:

[0218] 1. The present invention provides an automatic operator arrangement method and system based on a large model. The large model automatically handles the adaptability problems of interfaces and parameters through operator warehouse learning, reduces the complexity, and thus makes the arrangement of algorithm workflows more efficient and targeted.

[0219] 2. The present invention provides an automatic operator arrangement method and system based on a large model. By only describing the requirements, the large language model can understand and extract key information, realize fast operator selection and workflow arrangement, and greatly reduce the user's learning cost and manual operation.

[0220] 3. The present invention provides an automatic operator orchestration method and system based on a large model. Through real-time feedback and modification from users, the large model can be continuously fine-tuned and optimized to make its understanding of user needs more accurate. Accordingly, the orchestrated workflow will be more in line with user expectations.

[0221] 4. The present invention provides an automatic operator arrangement method and system based on a large model, and also provides a test running and modification environment, so that users can immediately see the execution effect of the workflow and make necessary adjustments, thereby speeding up the launch of the project.

[0222] 5. The present invention provides an automatic operator arrangement method and system based on a large model. Each manual modification and feedback from the user is a learning opportunity for the large model, making it more and more mature in long-term applications and having higher response speed and accuracy for similar needs.

[0223] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for automatic operator arrangement based on a large model, characterized in that: The method comprises the following steps: Step S1, define an operator as the smallest unit of calculation, classify it by function and store it in an operator library; Step S2: Collect user requirements, operator content, and workflow data for fine-tuning the model, perform data cleaning, and integrate them into a fine-tuning data set; Step S3: Use the target task dataset to fine-tune the large model so that it can understand and orchestrate operators; Step S4: After the big model analyzes the user's needs and extracts key information, it selects appropriate operators from the operator warehouse and arranges workflows for the user; Step S5: Display the arranged workflow to the user, collect their feedback, and perform iterative optimization as needed; Step S6: Monitor workflow execution in real time, provide optimization suggestions for workflows, generate execution reports, and use feedback and data to continuously fine-tune the large model.

2. The method for automatic operator arrangement based on a large model as claimed in claim 1, characterized in that: The step S2 specifically includes: Step S21: obtaining text from users describing their computing needs through questionnaires, interviews, or online submissions; Step S22: extracting the operator content of each operator from the operator library, including: detailed description, code, and input and output formats; Step S23: Collect complete workflow data, including user requirements, selected operators, and execution results; Step S24: Use a regular expression tool to remove unnecessary characters or strings from user requirements, operator content, and workflow data; Step S25: using a common stop word library to remove non-critical words; Step S26: unify the vocabulary into its basic form using NLP tools; Step S27: Use the data processing library to integrate and format the data to form a fine-tuning data set to meet the needs of model fine-tuning.

3. The method for automatic operator arrangement based on a large model as claimed in claim 1, characterized in that: The step S3 specifically includes: Step S31: Determine the batch size, learning rate and number of iterations for fine-tuning; Step S32: data input, dividing the integrated fine-tuning data set into a training set and a validation set; Step S33: Use the training set to fine-tune the large model, and use the validation set to check the performance of the model.

4. The method for automatic operator arrangement based on a large model as claimed in claim 1, characterized in that: The step S4 specifically includes: Step S41: using deep learning or NLP technology to parse the text input by the user and identify its core intent and key information; Step S42: Use the large model trained in S33 to score the function of each operator and select the most suitable operator for the user's needs; Step S43: Organize the selected operators into a complete workflow according to the order and logic of user requirements.

5. The method for automatic operator arrangement based on a large model as claimed in claim 1, characterized in that: The step S5 specifically includes: Step S51: providing a graphical interface for the user to display the recommended workflow structure; Step S52: Allow the user to modify the workflow or provide feedback on the interface; Step S53: Based on user feedback, the large model re-performs operator matching and workflow orchestration.

6. The method for automatic operator arrangement based on a large model as claimed in claim 1, characterized in that: The step S6 specifically includes: Step S61: deploy monitoring tools to track the execution of the workflow; Step S62: regularly collect workflow execution time, error rate, and resource consumption data; Step S63: Feed the collected performance data back to the large model to further optimize and fine-tune the large model.

7. The method for automatic operator arrangement based on a large model as claimed in claim 6, characterized in that: The S63 includes: Step S6311: Establish a monitoring system to track updates to the operator library and changes in user input patterns; Step S6312: Generate a fine-tuning trigger mechanism to automatically start fine-tuning when a significant change is detected; Step S6313: After each interaction between the user and the model, collect and analyze the user's feedback; Step S6314: Perform small-scale fine-tuning on the model based on the feedback.

8. The method for automatic operator arrangement based on a large model as claimed in claim 6, characterized in that: The S63 also includes user interactive iterative feedback, specifically including: Step S6321: Design an intuitive interface so that users can see the workflow generated for them by the big model; Step S6322: providing a real-time chat window on the interface to allow the user to interact with the large model; Step S6323: When the user asks or provides feedback, the large model should respond immediately; Step S6324: For each question or feedback from the user, the big model should provide a reasonable answer or make corresponding adjustments.

9. The method for automatic operator arrangement based on a large model as claimed in claim 6, characterized in that: The S63 also includes model interpretability enhancements, specifically including: Step S6331: When the model makes a selection for the user, an explanation text is automatically generated; Step S6332: The explanation should clearly describe why the model makes such a choice; Step S6333: On the user interface, there should be a "Why?" button next to each model selection; Step S6334: When the user clicks, the explanation provided by the model should be displayed immediately.

10. An operator automatic arrangement system based on a large model, characterized in that: The system includes: an input module, a conversion module, a fine-tuning module, an interaction module, an interpretation module, a background management module and an output module; The input module comprises: Input interface: allows users to select input methods, including text, voice, image, and video; File processing: ensuring that the content uploaded by users is in the correct format and setting size and format restrictions; Preprocessing tools: convert speech into text, identify text or key information in images or videos; The conversion module comprises: CLIP adapter: Use CLIP model to parse non-text input; Data matching: Match with text data to find the most relevant description; The fine-tuning module comprises: Monitoring tools: real-time tracking of operator library updates and user input patterns; Online learner: collects user feedback and performs small-scale fine-tuning in real time; The interaction module includes: Workflow display: Display the workflow generated by the model to the user; Live conversation interface: A real-time chat window that allows users to interact with the model; Feedback processor: adjusts the model based on user feedback; The interpretation module comprises: Explanation Generator: Generates explanation text for each choice of the model; Interface display tool: Display explanations on the interface and provide interactive buttons for users to query; The background management module includes: Database management: store and manage operator libraries, user input and feedback; Performance monitoring: monitor the performance of models and systems; Update scheduling: Update the system based on performance data and feedback; The output module comprises: Results display: The final workflow design is presented to the user; Error Reporting: Generates error reports when problems occur and provides resolution suggestions.

Images