Trusted artificial intelligence system taking tool large model as core and processing method
The trusted AI system with a tool big model core addresses the opacity and inaccuracy of large language models by using perception and decision modules to rely on stored tools and workflows, improving interpretability and reliability for industrial applications.
Patent Information
- Application Number
- CN202411158044.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2025-01-03
AI Technical Summary
Current large language models, such as GPT, are complex and opaque, leading to difficulty in understanding their decision-making processes and generating inaccurate or fictitious information, limiting their adoption in industrial applications.
A trusted artificial intelligence system with a tool big model core that enhances interpretability and transparency by using perception, decision, tool making, and tool planning modules to rely on stored tools and workflows, avoiding model opacity and improving accuracy.
The system provides interpretable and robust decision-making, facilitating human-machine collaboration and easy verification, enhancing the reliability and flexibility of large language models for industrial applications.
Smart Images
Figure CN119250196A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and more specifically to a trusted artificial intelligence system and processing method with a tool big model as the core. Background Art
[0002] The current mainstream artificial intelligence research focuses on machine learning methods based on deep learning, among which large language models represented by GPT are particularly prominent. Such models have hundreds of millions of parameters, strong generalization capabilities, and can perform well in a variety of language tasks. However, due to the complex and opaque internal structure of these models, they are often called "black boxes" and it is difficult for users to understand their decision-making process. At the same time, the model may also generate inaccurate or fictitious information, which is called the "hallucination" phenomenon, further limiting its widespread adoption in actual industrial applications.
[0003] Therefore, it is necessary to explore a trusted mechanism so that it can be effectively controlled and trusted by users during the application process. Summary of the invention
[0004] In view of this, based on the above problems, the present invention provides a trusted artificial intelligence system and processing method with a large tool model as the core, which can enhance the interpretability and transparency of the model, ensure the accuracy and consistency of the results, and promote the application of large language models in various industries, thereby releasing their potential value and alleviating the constraints of existing problems on practical applications.
[0005] In order to achieve the above object, the present invention adopts the following technical solution:
[0006] In a first aspect, an embodiment of the present invention provides a trusted artificial intelligence system with a tool big model as the core, including:
[0007] The perception module is used to perceive environmental information, collect and analyze environmental data after task input, so as to provide decision-making for the decision model module;
[0008] A decision model module determines whether to call the tool making large model module to generate a new tool or use the tool planning large model module to select an existing tool based on the environmental information and task input obtained by the perception module;
[0009] The tool making large model module is used to receive environmental information and task input, retrieve the tool warehouse module and workflow warehouse module, refer to existing tools and processes according to the task definition, and generate new tools;
[0010] The tool planning model is used to select the corresponding tool from the tool warehouse module to complete the task according to the environment information and task input;
[0011] The tool warehouse module is used to store tools generated manually or by the tool planning large model module; the tools include: software modules, physical devices or virtual tools;
[0012] The workflow warehouse module is used to store fixed workflows and record the process of environment perception, tool selection and use.
[0013] Furthermore, the environmental data is the information required by the carrier when the model system completes the task, which is represented by E as a multidimensional vector, where each dimension corresponds to a perceived feature; it is represented as:
[0014] E=[e1,e2,…,e n ] (1)
[0015] Among them, e i represents a certain perceptual feature; n represents the number of perceptual features.
[0016] Furthermore, the perception module specifically uses a multimodal large model to convert the information carrying method in multiple different scenes into preset structured data for other modules to understand; wherein the preset structured data includes: image recognition results, scene understanding information, object relationship information and environmental status information.
[0017] Furthermore, the mathematical modeling of the decision model module is as follows:
[0018] Let the environment information be E, the task input be T, and the tool set be Where TLC i represents the tool candidate; the decision function D is defined as:
[0019]
[0020] Among them, f(·) represents the comprehensive evaluation function of environmental information and task input, and θ represents the decision threshold.
[0021] Furthermore, the mathematical modeling of the tool making large model module is as follows:
[0022] The tool generation function G is defined as:
[0023]
[0024] in, is the candidate tool set, M is the total number of tools; TL1 is the newly generated tool.
[0025] Furthermore, the tool makes a large model module, specifically including:
[0026] Analysis and determination unit, used to analyze environmental information and task descriptions to determine the functional and performance requirements of the required tools;
[0027] A screening unit, used for screening qualified tools from the tool warehouse module according to the tool selection criteria;
[0028] Determine the combination unit, which is used to determine how to combine the selected tools into new tools according to the tool combination plan to meet the task requirements;
[0029] The output definition unit is used to output the definition of the new tool, including how to compose the existing tools, as well as the specific operation steps and expected output of the new tool.
[0030] Furthermore, the mathematical modeling of the tool planning large model is as follows:
[0031] The tool selection function S is defined as:
[0032]
[0033] in, is a tool set and TL2 is the selected tool.
[0034] Furthermore, the tool plans a large model, specifically including:
[0035] An analysis unit, used to analyze environmental information and task descriptions to determine task requirements and environmental conditions;
[0036] A selection unit is used to select corresponding tools from the tool warehouse module or make purchase suggestions according to the tool selection criteria;
[0037] Development unit, which is used to develop a task execution plan, describing the tools to be used and the expected results for each step;
[0038] The output unit is used to output the task execution plan, including the selected tools and step sequence.
[0039] Furthermore, the mathematical modeling of the workflow warehouse module is as follows:
[0040] The set W of recorded workflows is defined as:
[0041] W={(E0,T0,TL0,R0),(E1,T1,TL1,R1),…,(E n ,T n ,TL n ,R n )} (5)
[0042] Among them, E i Indicates environmental information, T i represents the task input, TL i Indicates the tool used, R i Indicates the result.
[0043] In a second aspect, an embodiment of the present invention further provides a processing method of a trusted artificial intelligence system with a tool big model as the core, using the trusted artificial intelligence system as described in any one of the first aspects, and the processing method includes the following steps:
[0044] Get the input task; perceive the environment according to the input task perception module and convert it into environmental information E and task T;
[0045] The decision model module determines to call the tool making large model module or the tool planning large model module according to the environmental information E and the task T;
[0046] The tool making big model module generates new tools based on the environment information E and the task T, or the tool planning big model module selects existing tools from the tool warehouse module;
[0047] Use the generated new tools or select existing tools to complete the task and obtain the results;
[0048] The perception module evaluates the results, updates the environment information and tasks, and records them into the workflow warehouse module.
[0049] It can be seen from the above technical solution that compared with the prior art, the present invention has the following advantages:
[0050] The present invention is different from general artificial intelligence systems with large language models as the core. During the reasoning process, it gives priority to relying on tools and workflows stored in the tool warehouse and workflow warehouse to execute tasks, thereby avoiding the illusion of the large model itself and the high reasoning cost.
[0051] Moreover, the workflow of the artificial intelligence system shows the entire workflow and logic to the user, which is highly interpretable and can easily facilitate human-machine collaboration. It can also be more easily verified and corrected manually, further enhancing its robustness. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0053] Figure 1 This is a schematic diagram of the system modules and workflow provided by the present invention.
[0054] Figure 2 A schematic diagram of reasoning using a workflow provided by the present invention.
[0055] Figure 3 This is a schematic diagram of the workflow warehouse module provided by the present invention.
[0056] Figure 4 A flow chart of the processing method of the trusted artificial intelligence system with the tool big model as the core provided by the present invention.
[0057] Figure 5 A training diagram for making a large model module using the tool provided by the present invention. DETAILED DESCRIPTION
[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0059] Reference Figure 1 As shown, the embodiment of the present invention discloses a trusted artificial intelligence system with a tool big model as the core, which includes the following six modules: a perception module, a decision model module, a tool production big model module, a tool planning big model module, a tool warehouse module and a workflow warehouse module; Figure 2 As shown, a schematic diagram of the system using workflow for reasoning.
[0060] The following is a detailed description of each of the above modules:
[0061] 1. Perception module
[0062] The perception module is responsible for perceiving environmental information, collecting and parsing environmental data after task input, so that the decision-making model can make decisions. The environment here refers to the carrier of the information required by the model system when completing the task. The environmental information E can be expressed as a multidimensional vector, where each dimension corresponds to a perceived feature. For example, for the home life assistant scenario, the environmental information can be expressed as:
[0063] E=[e1,e2,…,e n ] (1)
[0064] Among them, e i Represents a certain perceptual feature, such as temperature, light intensity, sound level, etc. n represents the number of perceptual features.
[0065] For example:
[0066] Code development scenario: In this scenario, "environment" can refer to the current code development interface and development environment, and the "perception module" is used to obtain the code in the current code development interface and the code running status and results fed back by the development environment.
[0067] Computer desktop assistant scenario: In this scenario, "environment" can refer to the current computer desktop environment and the peripherals that can be interacted with, and the "perception module" is used to obtain the display of the current desktop environment and the operation of the peripherals and the subsequent feedback.
[0068] Home life assistant scenario: In this scenario, "environment" can refer to the entire indoor environment that can be sensed by sensors such as image, sound, and temperature. The "perception module" is used to use these sensors to perceive the current indoor state and interaction with the user.
[0069] Industrial production scenario: The perception module monitors the status of the production line in real time, including machine operation and product quality, through cameras, pressure sensors, temperature sensors and other equipment.
[0070] Medical assistance scenario: The perception module collects the patient's physiological data, such as heart rate, blood pressure, blood sugar, etc., through medical equipment, and combines it with information such as electronic medical records for environmental perception.
[0071] The perception module can use sensors, cameras, microphones, API interfaces and other tools to collect environmental data. These tools convert raw data into structured environmental descriptions through signal processing, data parsing and other methods.
[0072] Implementation of the perception module:
[0073] An implementation example of the perception module is to use a multimodal large model. Here, the multimodal large model LLaVA is selected to support the conversion of information carrying methods in different scenarios such as vision, hearing, and text into a form that can be understood by subsequent models such as decision models. Here, an implementation example of this form is a structured data.
[0074] Perception module data format design:
[0075] Environmental information content:
[0076] Image recognition results: Identify objects in the image and their attributes, such as type, size, color, etc.
[0077] Scene understanding: describing the scene in the image, such as indoor, outdoor, workplace, etc.
[0078] Object relationship: describes the spatial relationship between objects, such as position, direction, relative distance, etc.
[0079] Environmental status: describes the dynamic changes of the environment, such as the flow of people and vehicles.
[0080] Data field definition:
[0081] timestamp: The timestamp when the data is generated.
[0082] version: The version of the data format.
[0083] image_data: A reference or summary of the original image data.
[0084] object_list: A list of objects identified. Each object contains the following subfields:
[0085] object_id: Unique identifier.
[0086] type: object type.
[0087] attributes: attributes of an object, such as size, color, etc.
[0088] position: The position coordinates of the object in the image.
[0089] confidence: Recognition confidence.
[0090] scene_description: Description of the scene, including scene type and characteristics.
[0091] spatial_relationships: Description of the spatial relationships between objects.
[0092] dynamic_state: Description of the dynamic state of the environment.
[0093] 2. Decision model module
[0094] The decision model decides whether to call the tool making big model to generate a new tool, or use the tool planning big model to select an existing tool, based on the environmental information and task input obtained by the perception module.
[0095] Mathematical modeling of decision-making models:
[0096] Let the environment information be E, the task input be T, and the tool set be Where TLC i represents the tool candidate; the decision function D is defined as:
[0097]
[0098] Among them, f(·) represents the comprehensive evaluation function of environmental information and task input, and θ represents the decision threshold.
[0099] Data field definition
[0100] timestamp: The timestamp when the data is generated.
[0101] version: The version of the data format.
[0102] task_input: Detailed information describing the task requirements, including task type, objectives, constraints, etc.
[0103] environment_summary: A summary of the environment information provided by the perception module.
[0104] decision_parameters: Parameters used by the decision model, such as thresholds, priorities, etc.
[0105] action: The decision result clearly indicates whether to call the tool to make a large model or to plan a large model, and provides the necessary calling parameters.
[0106] Instructions
[0107] The task input (task_input) should describe the task requirements in detail so that the decision model can accurately understand the task goals.
[0108] The environment summary (environment_summary) should provide a summary of the key information of the perception module for reference by the decision model.
[0109] The decision parameters (decision_parameters) should include any thresholds or priority settings used in the decision process.
[0110] The action should clearly indicate the decision result, including the selected module and necessary calling parameters.
[0111] 3. Tool making large model module
[0112] The tool making model is used to generate specific tools according to the task definition. The model accepts the environment information E and the task T, retrieves the tool warehouse and workflow warehouse, refers to the existing tools and processes, and generates new tools.
[0113] Mathematical modeling of tool making large models:
[0114] The tool generation function G is defined as:
[0115]
[0116] in, is the candidate tool set, M is the total number of tools; TL1 is the newly generated tool.
[0117] Data encapsulation structure
[0118] Header: Contains basic information about the data, such as version number, data type, timestamp, etc.
[0119] EnvironmentInfo: Environment information, including available resources, tools, etc.
[0120] TaskDescription: A detailed description of the task requirements.
[0121] ToolSelectionCriteria: The criteria or logic for selecting an existing tool.
[0122] ToolCombinationPlan: describes how to combine existing tools to generate new tools.
[0123] NewToolSpecification: The definition of the new tool, including features, inputs and outputs, etc.
[0124] Footer: Contains the checksum of the data, end mark, etc.
[0125] Data field definition
[0126] timestamp: The timestamp when the data is generated.
[0127] version: The version of the data format.
[0128] environment_info: Environment information, including available materials, tool lists, etc.
[0129] task_description: Describe the task requirements in detail, including task objectives, performance indicators, etc.
[0130] tool_selection_criteria: Criteria for selecting tools, such as functional requirements, compatibility, etc.
[0131] tool_combination_plan: A detailed plan for combining existing tools to generate a new tool.
[0132] new_tool_specification: The definition of the new tool, including name, function, input and output parameters, etc.
[0133] Instructions
[0134] Environment information (environment_info) should provide a list of currently available tools and materials, helping the model understand what resources are available.
[0135] The task description (task_description) should describe the task objectives and performance indicators in detail to guide the selection and combination of tools.
[0136] The tool selection criteria (tool_selection_criteria) should clearly define the functional requirements and compatibility criteria for selecting tools.
[0137] Tool combination plan (tool_combination_plan) describes how to select and combine existing tools according to task requirements.
[0138] The new tool specification (new_tool_specification) should define the name, function, input and output parameters, and component composition of the new tool in detail.
[0139] Generate Logic
[0140] The tool making model first analyzes the environmental information and task description to determine the functional and performance requirements of the required tools.
[0141] According to the tool selection criteria, the model filters the qualified tools from the tool warehouse.
[0142] Based on the tool combination plan, the model determines how to combine the selected tools into new tools to meet the task requirements.
[0143] Finally, the model outputs the definition of the new tool, including how it is composed from existing tools, as well as the specific operating steps and expected outputs of the new tool.
[0144] Through this structured instruction data format, the tool making big model can clearly receive task and environment information, and carry out effective tool design and generation based on it, including combining existing tools to meet specific task requirements.
[0145] 4. Tool planning large model module
[0146] The tool planning model selects appropriate tools from the tool warehouse to complete the task based on the environmental information E and the task T.
[0147] Mathematical modeling of tool planning large model:
[0148] The tool selection function S is defined as:
[0149]
[0150] in, is a tool set and TL2 is the selected tool.
[0151] Data encapsulation structure
[0152] Header: Contains basic information about the data, such as version number, data type, timestamp, etc.
[0153] EnvironmentInfo: Environmental information, including resources, conditions, etc.
[0154] TaskDescription: A detailed description of the task requirements.
[0155] ToolSelectionCriteria: The criteria or logic for selecting a tool.
[0156] SelectedTools: List of tools selected according to the selection criteria.
[0157] TaskExecutionPlan: Task execution plan, including the steps and sequence of using tools.
[0158] Footer: Contains the checksum of the data, end mark, etc.
[0159] Data field definition
[0160] timestamp: The timestamp when the data is generated.
[0161] version: The version of the data format.
[0162] environment_info: Environmental information, including available tools, materials, environmental restrictions, etc.
[0163] task_description: A detailed description of the task requirements, including task objectives, performance requirements, time limits, etc.
[0164] tool_selection_criteria: Criteria for selecting tools, such as functional matching, efficiency, durability, etc.
[0165] selected_tools: A list of tools selected based on the selection criteria and task requirements, including tool IDs and brief descriptions.
[0166] task_execution_plan: A detailed task execution plan, including the tools used and expected results for each step.
[0167] Instructions
[0168] Environment information (environment_info) should provide a list of currently available tools and other environmental factors that may affect tool selection.
[0169] The task description (task_description) should describe the task objectives and performance requirements in detail to guide the selection of tools.
[0170] Tool selection criteria (tool_selection_criteria) should clearly identify factors to consider when selecting a tool, such as functionality, efficiency, and durability.
[0171] Selected_tools should list the tools selected based on the criteria and explain why they were selected.
[0172] The task execution plan (task_execution_plan) should describe each step of the task in detail, including the tools used and the expected output.
[0173] Planning Logic
[0174] The tool planning model first analyzes the environmental information and task description to determine the task requirements and environmental conditions.
[0175] Based on the tool selection criteria, the model selects suitable tools from the tool warehouse or makes purchase recommendations.
[0176] The model develops a plan for executing the task, detailing the tools to be used and the expected results for each step.
[0177] Finally, the model outputs a task execution plan, including the selected tools and the detailed sequence of steps.
[0178] Through this structured instruction data format, the tool planning model can clearly receive task and environment information, and make effective tool selection and task planning based on it to ensure the smooth completion of the task.
[0179] 5. Tool Warehouse
[0180] The tool warehouse stores tools generated manually or by tool planning large models. These tools can be software modules, physical devices or virtual tools that can perform specific atomic tasks.
[0181] Tool examples:
[0182] Optical Character Recognition (OCR)
[0183] Numerical calculator
[0184] Sequencer
[0185] Finder
[0186] The management of the tool warehouse includes operations such as storage, retrieval, and updating of tools to ensure the availability and effectiveness of the tools.
[0187] 6. Workflow warehouse module
[0188] Reference Figure 3 As shown in Figure 1, the workflow warehouse stores fixed workflows and records the process of environment perception, tool selection, and use. These records can be directly applied in specific scenarios to reduce the instability of large models.
[0189] Mathematical modeling of workflow warehouse:
[0190] The set W of recorded workflows is defined as:
[0191] W={(E0,T0,TL0,R0),(E1,T1,TL1,R1),…,(E n ,T n ,TL n ,R n )} (5)
[0192] Among them, E i Indicates environmental information, T i represents the task input, TL i Indicates the tool used, R i Indicates the result.
[0193] Reference Figure 4 As shown, the working process of the system is as follows:
[0194] S10, obtaining an input task; the environment is sensed by the input task perception module and converted into environmental information E and task T;
[0195] S20, the decision model module determines to call the tool making large model module or the tool planning large model module according to the environmental information E and the task T;
[0196] S30, the tool making large model module generates a new tool according to the environment information E and the task T, or the tool planning large model module selects an existing tool from the tool warehouse module;
[0197] S40, completing the task and obtaining the result according to the generated new tool or selecting the existing tool;
[0198] S50, the perception module evaluates the results, updates the environmental information and tasks, and records them into the workflow warehouse module.
[0199] Reference Figure 5 As shown, the production and training of the tool model:
[0200] The tool makes large models and uses different large models to adjust according to different scenarios. The following are specific examples:
[0201] Document analysis scenario: Use a large language model (such as GPT) as the basic model, input the scanned document image, and generate tools including OCR tools, numerical calculators, sorters, and finders.
[0202] Physical world scenario: Use 3D printers and other equipment to manufacture physical tools and store them in a physical tool warehouse.
[0203] The training data of the tool production model includes tool input, tool program and tool input and output triples. Through training, the large language model can generate corresponding tool composition codes for specific tasks.
[0204] Aiming at the actual application scenarios, this technology continues to improve its capabilities in two scenarios: games and code generation. The following are specific examples of building training data:
[0205] Games: Taking Minecraft as an example, the main idea is to combine mature models (such as GPT) and Minecraft’s own game feedback to build training data
[0206] In a game environment such as Minecraft, building training data needs to be designed around in-game objects, behaviors, and player interactions. The following are the steps to build training data, while considering how to use game run results to evaluate system performance:
[0207] Steps to build training data
[0208] 1. Define the game data structure:
[0209] Data structures that determine in-game objects (blocks, entities, tools, etc.), player behavior, and environment state.
[0210] Collecting in-game data:
[0211] Collect data from game logs, player operation records, or game APIs.
[0212] Perception module training data:
[0213] The environment state and object information within the game are used to generate training data for the perception module.
[0214] Decision model training data:
[0215] Create task descriptions containing different game scenarios and goals for training decision-making models.
[0216] Tools for making large model training data:
[0217] Generate or select combinations of in-game blocks and entities to simulate the creation of new tools or structures.
[0218] Tools plan large model training data:
[0219] Generate planning and workflow arrangements for tool usage in the game based on task requirements.
[0220] Tool warehouse and workflow warehouse training data:
[0221] A collection of tools, blocks, and workflow templates available in the game.
[0222] Generate in-game tasks and objectives:
[0223] Use GPT to generate possible tasks and goals in the game, such as building a specific structure or achieving a certain game achievement.
[0224] Simulate game running:
[0225] Simulate task execution in the game environment and collect game operation results and player feedback.
[0226] Performance feedback generation:
[0227] Based on the game running results, use GPT to generate system performance evaluation and feedback.
[0228] Data annotation:
[0229] The generated tasks, game running results and performance feedback are annotated to ensure the accuracy of the data.
[0230] Data verification and balancing:
[0231] Verify the accuracy and consistency of the data, ensure the balance of training data, and cover different game scenarios.
[0232] The dataset is split into training set, validation set, and test set for model training and evaluation.
[0233] Embodiment 1:
[0234] An example of using GPT to generate in-game training data for Minecraft
[0235] Perception module training data:
[0236] Enter GPT: "Generate descriptions ofvarious biomes,structures,andentities in Minecraft for the perception module."
[0237] Decision model training data:
[0238] Enter GPT: "Create scenarios where a playerneeds to decide between different building materials in Minecraftbased on the environment."
[0239] Tools for making large model training data:
[0240] Enter GPT: "Propose a new tool or structure inMinecraft that combines existing blocks to achieve a specific function."
[0241] Tools plan large model training data:
[0242] Enter GPT: "Outline aworkflow forbuilding a redstone circuit inMinecraft, including the selection of blocks and the assembly process."
[0243] Tool warehouse and workflow warehouse training data:
[0244] Collect blocks, tools, and redstone circuit designs available in Minecraft as training data.
[0245] Generate in-game tasks and objectives:
[0246] Enter GPT: "Design atask for a Minecraftplayer to create an automated farm using redstone and othergame elements."
[0247] Simulate game running:
[0248] Simulate players' tasks in the Minecraft environment and record the operation steps and final results.
[0249] Performance feedback generation:
[0250] Enter GPT: "Based on the success or failure of the automated farminMinecraft, generate feedback on the system's performance."
[0251] Iteration and optimization:
[0252] Based on performance feedback, GPT is used to generate improvement suggestions and iteratively optimize the training data.
[0253] In this way, a comprehensive training dataset can be built that covers the entire process from game perception, decision-making, tool making, planning to execution and feedback, thereby training an intelligent system that can adapt to game environments such as Minecraft.
[0254] Code generation scenario:
[0255] Steps to build training data
[0256] Define the training objective:
[0257] Clarify the goals that the training data needs to achieve, such as generating code snippets for specific functions, optimizing existing code tools, improving code performance, etc.
[0258] Collecting codebase data:
[0259] Collect basic code snippets from open source projects, code repositories, historical projects, etc. as the starting point for training data.
[0260] Perception module training data:
[0261] Leverage existing code bases and documentation to generate metadata that describes the code's functionality, usage, and dependencies.
[0262] Decision model training data:
[0263] Create task descriptions that include different programming scenarios, requirements, and constraints to train decision models to select appropriate code tools.
[0264] Tools for making large model training data:
[0265] Generate or select code snippets to simulate the creation process of new tools, including the writing of functions, classes, and libraries.
[0266] Tools plan large model training data:
[0267] Based on task requirements, generate a combination and usage plan of code tools, including tool selection and workflow arrangement.
[0268] Tool warehouse and workflow warehouse training data:
[0269] Collect and organize existing code tools and workflow templates to provide reference for the model.
[0270] Generate code combination logic:
[0271] Use GPT to generate the combinatorial logic of code tools, describing how to combine different code snippets or tools into new tools.
[0272] Simulate code execution:
[0273] Simulate code execution in a controlled environment to collect execution results, performance metrics, and potential errors.
[0274] Performance feedback generation:
[0275] Based on the code execution results, use GPT to generate performance feedback and improvement suggestions.
[0276] Data annotation and enhancement:
[0277] The generated code and execution results are annotated, including functional correctness, performance evaluation, etc., and data enhancement is performed to improve diversity.
[0278] Data verification and balancing:
[0279] Verify the accuracy and consistency of data, ensure the balance of training data, and cover different programming languages, tools, and scenarios.
[0280] Data segmentation:
[0281] The dataset is split into training set, validation set, and test set for model training and evaluation.
[0282] Embodiment 2:
[0283] An example of using GPT to generate code to generate training data
[0284] Perception module training data:
[0285] Enter GPT: "Generate metadata for a set of Python libraries used for data analysis."
[0286] Decision model training data:
[0287] Enter GPT: "Describe a scenario where a developer needs to choose between different machine learning libraries for a project."
[0288] Tools for making large model training data:
[0289] Enter GPT: "Create aPython functionthat implements a new data cleaningtool based on the provided criteria."
[0290] Tools plan large model training data:
[0291] Enter GPT: "Outline aworkflow for combining data cleaning, transformation, and visualization tools in aPython data analysis pipeline."
[0292] Tool warehouse and workflow warehouse training data:
[0293] Collect commonly used data science Python libraries and workflow templates as training data.
[0294] Generate code combination logic:
[0295] Enter GPT: "Explain how to integrate anewly created data cleaning function with existing transformation and visualization libraries."
[0296] Simulate code execution:
[0297] Execute the generated code portfolio in a Python environment, recording its performance and output.
[0298] Performance feedback generation:
[0299] Enter GPT: "Based on the execution results, generate feedback on the performance of the integrated data analysis tools."
[0300] Iteration and optimization:
[0301] Based on performance feedback, use GPT to generate improvement suggestions and iteratively optimize code tools.
[0302] In this way, a comprehensive training dataset can be constructed that covers the entire process from code perception, decision-making, production, planning to execution and feedback, thereby training a system that can generate, combine and optimize code tools.
[0303] Practical application scenarios
[0304] 1). Software development scenario:
[0305] Scenario description: In a software development project, developers need to resolve code errors and optimize performance.
[0306] System workflow:
[0307] 1. Enter a task description (e.g., "Resolve a coding error").
[0308] 2. The perception module obtains the current code and its error information.
[0309] 3. The decision model decides to call the tool planning model based on the error information and task description.
[0310] 4. Tool planning Choose appropriate debugging tools (such as error detectors) for large models.
[0311] 5. Use debugging tools to detect and fix code errors and obtain repair results.
[0312] 6. Update the environment information and task description and record them in the workflow warehouse.
[0313] 2). Smart home scenarios:
[0314] Scenario description: The user wants to control the lights and temperature in the home by voice.
[0315] System workflow:
[0316] 1. Enter a task description (e.g., "Adjust the lighting and temperature").
[0317] 2. The perception module obtains the current indoor environment information (such as lighting status and temperature).
[0318] 3. The decision model decides to use tools to create a large model based on environmental information and task description.
[0319] 4. Tool making: Generate corresponding control tools (such as light controller, temperature regulator) from large models.
[0320] 5. Use the generated control tools to adjust the light and temperature and obtain the adjustment results.
[0321] 6. The perception module evaluates the adjustment results, updates the environmental information and task description, and records them into the workflow warehouse.
[0322] 3). Industrial production scenarios:
[0323] Scenario description: In a smart manufacturing factory, it is necessary to optimize the operating efficiency of the production line and monitor product quality.
[0324] System workflow:
[0325] 1. Enter a task description (e.g., "Optimize production line efficiency").
[0326] 2. The perception module obtains real-time status information of the production line through cameras and sensors.
[0327] 3. The decision model decides to call the tool planning model based on the production line status information and task description.
[0328] 4. Tool planning The big model selects appropriate optimization tools (such as production scheduling optimizer, quality monitor).
[0329] 5. Use optimization tools to adjust production line operations, monitor product quality, and obtain optimization results.
[0330] 6. The perception module evaluates the optimization results, updates the environment information and task description, and records them into the workflow warehouse.
[0331] The trusted artificial intelligence system with a large tool model as the core provided by the present invention has the capabilities of efficient perception, flexible decision-making, dynamic generation and selection of tools, and continuous optimization, which greatly improves the intelligence level and practicality of the system. In addition, the system and processing method provided by the present invention not only solve the problems of uncontrollable, unexplainable and unreliable models in the prior art, but also enhance the flexibility, adaptability and user trust of the system, thereby promoting the widespread adoption and value realization of large language models and other deep learning technologies in industrial applications.
[0332] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0333] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A trusted artificial intelligence system with a tool big model as the core, characterized by: include: The perception module is used to perceive environmental information, collect and analyze environmental data after task input, so as to provide decision-making for the decision model module; A decision model module determines whether to call the tool making large model module to generate a new tool or use the tool planning large model module to select an existing tool based on the environmental information and task input obtained by the perception module; The tool making large model module is used to receive environmental information and task input, retrieve the tool warehouse module and workflow warehouse module, refer to existing tools and processes according to the task definition, and generate new tools; The tool planning model is used to select the corresponding tool from the tool warehouse module to complete the task according to the environment information and task input; The tool warehouse module is used to store tools generated manually or by the tool planning large model module; the tools include: software modules, physical devices or virtual tools; The workflow warehouse module is used to store fixed workflows and record the process of environment perception, tool selection and use.
2. According to claim 1, a trusted artificial intelligence system with a tool big model as the core is characterized in that: The environmental data is the information that the model system needs to carry when completing the task, and is represented by E as a multidimensional vector, where each dimension corresponds to a perceived feature; it is represented as: E=[e1,e2,…,e n ] (1) Among them, e i represents a certain perceptual feature; n represents the number of perceptual features.
3. A trusted artificial intelligence system with a tool big model as the core according to claim 1 or 2, characterized in that: The perception module specifically uses a multimodal large model to convert the information carrying method in multiple different scenes into preset structured data for other modules to understand; wherein the preset structured data includes: image recognition results, scene understanding information, object relationship information and environmental status information.
4. According to claim 1, a trusted artificial intelligence system with a tool big model as the core is characterized in that: The mathematical modeling of the decision model module is as follows: Let the environment information be E, the task input be T, and the tool set be Where TLC i represents the tool candidate; the decision function D is defined as: Among them, f(·) represents the comprehensive evaluation function of environmental information and task input, and θ represents the decision threshold.
5. According to claim 4, a trusted artificial intelligence system with a tool big model as the core is characterized in that: The mathematical modeling of the tool making large model module is as follows: The tool generation function G is defined as: in, is the candidate tool set, M is the total number of tools; TL1 is the newly generated tool.
6. A trusted artificial intelligence system with a tool big model as the core according to claim 1, characterized in that: The tool makes a large model module, specifically including: Analysis and determination unit, used to analyze environmental information and task descriptions to determine the functional and performance requirements of the required tools; A screening unit, used for screening qualified tools from the tool warehouse module according to the tool selection criteria; Determine the combination unit, which is used to determine how to combine the selected tools into new tools according to the tool combination plan to meet the task requirements; The output definition unit is used to output the definition of the new tool, including how to compose the existing tools, as well as the specific operation steps and expected output of the new tool.
7. A trusted artificial intelligence system with a tool big model as the core according to claim 4, characterized in that: The mathematical modeling of the tool planning model is as follows: The tool selection function S is defined as: in, is a tool set and TL2 is the selected tool.
8. The trusted artificial intelligence system with a tool big model as the core according to claim 1 is characterized in that: The tool plans a large model, specifically including: An analysis unit, used to analyze environmental information and task descriptions to determine task requirements and environmental conditions; A selection unit is used to select corresponding tools from the tool warehouse module or make purchase suggestions according to the tool selection criteria; Development unit, which is used to develop a task execution plan, describing the tools to be used and the expected results for each step; The output unit is used to output the task execution plan, including the selected tools and step sequence.
9. The trusted artificial intelligence system with a tool big model as the core according to claim 1 is characterized in that: The mathematical modeling of the workflow warehouse module is as follows: The set W of recorded workflows is defined as: W={(E0,T0,TL0,R0),(E1,T1,TL1,R1),…,(E n ,T n ,TL n ,R n )} (5) Among them, E i Indicates environmental information, T i represents the task input, TL i Indicates the tool used, R i Indicates the result.
10. A processing method for a trusted artificial intelligence system with a tool big model as the core, characterized in that: Using the trusted artificial intelligence system as described in any one of claims 1 to 9, the processing method comprises the following steps: Get the input task; perceive the environment according to the input task perception module and convert it into environmental information E and task T; The decision model module determines to call the tool making large model module or the tool planning large model module according to the environmental information E and the task T; The tool making big model module generates new tools based on the environment information E and the task T, or the tool planning big model module selects existing tools from the tool warehouse module; Use the generated new tools or select existing tools to complete the task and obtain the results; The perception module evaluates the results, updates the environment information and tasks, and records them into the workflow warehouse module.