A visual algorithm self-training method based on multi-agent collaborative optimization

Abstract

The application discloses a kind of visual algorithm self-training methods based on multi-agent collaborative optimization, comprising the following steps: constructing multi-agent system architecture, including user interaction layer, intelligent scheduling layer, A2A protocol communication layer and professional agent cluster layer;User interaction layer analyzes user task intention and generates execution plan;Scheduling agent calls professional agent to complete data processing, model building, training, testing and deployment;Task flow is coordinated through standardized communication mechanism, combined with MCP tool set, knowledge base module and memory system support task execution;When task fails, automatically execute rescheduling operation, finally output self-training result.The application significantly improves development efficiency, adaptability and intelligent level, and is suitable for industrial detection, intelligent security, autonomous driving and other computer vision tasks.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a self-training method for visual algorithms based on multi-agent collaborative optimization. Background Technology

[0002] Traditional development and application of small visual models require specialized human division of labor, and their workflow is highly rigid and inflexible. Specifically, the process begins with data labelers manually annotating large amounts of visual data. Algorithm engineers then research and design dedicated visual network models, AI trainers adjust hyperparameters and training strategies, deployment engineers embed the trained model into the actual hardware environment, and finally, visual application engineers integrate the visual model with specific business logic using tools such as OpenCV. This model is cumbersome, lengthy, and heavily reliant on the experience of experts in each domain, resulting in long development cycles, high labor costs, and difficulty in quickly adapting to continuously changing application requirements.

[0003] To improve efficiency, existing technologies typically use predefined workflows to connect the aforementioned steps. However, such workflows are inherently static and rigid, with their execution logic, parameters, and paths fixed during the development phase. When faced with new data distributions, task objectives, or unforeseen scenarios, these systems lack dynamic semantic understanding and autonomous decision-making capabilities. Developers must redesign the process, modify the code, and adjust parameters, failing to achieve true adaptive optimization and exhibiting severely insufficient robustness and flexibility. Traditional workflows involve repetitive, static processes with each execution, unable to learn and optimize based on actual performance. Consequently, they lack the ability to accumulate and utilize historical experience, resulting in poor performance in complex and ever-changing application scenarios, high maintenance costs, and an inability to quickly adapt to changes.

[0004] With the rise of Large Language Models (LLMs) technology, their powerful semantic understanding, task planning, and code generation capabilities have provided core impetus for building a new generation of adaptive systems. In recent years, LLMs supporting tool invocation have matured, making it possible to dynamically schedule external functional modules through natural language commands. Building on this, technical standards and frameworks such as Model Context Protocol (MCP) and Agent-to-Agent Protocols (A2A) have emerged, laying a solid foundation for building multi-agent systems capable of complex collaboration. This marks a paradigm shift from static workflows to dynamic agent collaboration, providing a completely new technological path to break through the limitations of traditional automated processes.

[0005] While applying large language models and multi-agent systems to general automated processes has become an important trend, their deep application in the end-to-end self-training process of small visual models remains an underexplored area. Existing solutions either mechanically string together existing steps in a workflow or only utilize large language models for partial steps (such as data augmentation or parameter recommendation), failing to fundamentally build a closed-loop system capable of autonomously understanding tasks, dynamically planning training strategies, coordinating multi-agent execution, and continuously optimizing based on feedback. Therefore, given the growing demand from enterprises for automated and intelligent development tools, there is an urgent need for a self-training method that fully leverages the advantages of multi-agent collaboration in large language models to completely replace rigid traditional workflows and achieve efficient, adaptive, and unattended automated training and continuous optimization of small visual models.

[0006] Therefore, how to provide a self-training method for visual algorithms based on multi-agent collaborative optimization is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] One objective of this invention is to propose a self-training method for visual algorithms based on multi-agent collaborative optimization. This invention fully integrates the integrated LLM model, the A2A protocol communication layer to establish a standardized communication mechanism, and the MCP toolset. Through a multi-agent system architecture, it realizes intelligent collaborative processing of the entire process of visual algorithm task parsing, task decomposition, sub-task allocation, tool invocation, and model construction. It describes in detail the self-training process of user task intent recognition, task execution plan construction, professional agent sub-task execution, standardized communication invocation, knowledge support, and memory maintenance. It has the advantages of high collaborative efficiency, complete task execution closed loop, flexible system expansion, and strong fault scheduling capability.

[0008] A self-training method for a visual algorithm based on multi-agent cooperative optimization according to an embodiment of the present invention includes the following steps:

[0009] S1. Construct a multi-agent system architecture; the multi-agent system includes a user interaction layer, an intelligent scheduling layer, an A2A protocol communication layer, and a professional agent cluster layer;

[0010] S2. In the user interaction layer, receive image or video data uploaded by the user, and parse and generate the user's task intent through the user interface that integrates the LLM model.

[0011] S3. The intelligent scheduling layer receives the user's task intent, constructs professional capability context information through the scheduling intelligent agent in the professional intelligent agent, and integrates the LLM model to complete task understanding and task decomposition, generates execution steps and dependencies, and forms an execution plan.

[0012] S4. Based on the execution plan, the scheduling agent calls upon specialized agents to execute corresponding sub-tasks;

[0013] S5. Establish a standardized communication mechanism in the A2A protocol communication layer to complete asynchronous calls, state updates, skill discovery and error handling between intelligent agents;

[0014] S6. In the professional intelligent agent cluster layer, the role definition module sets the professional division of labor, the LLM model executes the sub-task understanding and collaboration plan, the MCP toolset completes data processing and training, the knowledge base module provides domain knowledge support, and the memory system maintains the task context and interaction records.

[0015] S7. If any execution step fails during task execution, the scheduling agent performs a rescheduling operation; based on the task execution and multi-agent system response process, the self-training process is completed and the results are output.

[0016] Optionally, S2 specifically includes:

[0017] The user interaction layer receives image or video data uploaded by users and performs format recognition and input registration on the received data.

[0018] The task description text associated with the input data is extracted through the user interface that integrates the LLM model;

[0019] Based on the task description text, perform task intent recognition to generate task intent information; then construct the task intent information and input data into structured content for the task request.

[0020] The structured content of the task request is submitted to the intelligent scheduling layer, and the task number, input data index, and task submission time information are recorded.

[0021] Optionally, the intelligent scheduling layer includes:

[0022] The scheduling agent receives structured content of task requests, establishes a task number index table, and registers task priority, task type, and input data source.

[0023] The scheduling agent summarizes historical task records, professional agent role configurations, and knowledge base capability entries to generate professional capability context information.

[0024] By integrating an LLM model, the structured content of task requests is analyzed to understand the task objectives, processing objects, and output format information.

[0025] Based on task understanding, the task is decomposed by integrating an LLM model to generate a sequence of subtasks, a dependency table, and a resource requirement list, which constitutes the initial draft of the execution plan.

[0026] The scheduling agent performs structural verification and resource matching on the initial draft of the execution plan, marks the executable status, and generates the execution plan.

[0027] Optionally, the LLM model is specifically:

[0028] The LLM model serves as an inference engine, receiving structured content from task requests.

[0029] Based on the structured content of the task request, the task understanding is performed, and based on the task description text, the task objectives, processing objects and output requirements are identified to form semantic understanding results.

[0030] Based on the semantic understanding results, intelligent task decomposition is performed to generate a sub-task sequence, a dependency table, and a resource requirement list;

[0031] Based on the subtask sequence, the agent invocation strategy is analyzed to identify the required professional capabilities and select a matching agent combination to construct a multi-agent collaborative process; the LLM model is suitable for multi-agent collaborative optimization.

[0032] Dynamic task scheduling is adjusted based on agent state and resource availability, task allocation scheme is updated, and optimization feedback information is provided.

[0033] Optionally, the A2A protocol communication layer specifically includes:

[0034] A standardized communication mechanism is established at the A2A protocol communication layer to support message exchange and command transmission between scheduling agents and specialized agents; the specialized agents include data labeling agents, data analyst agents, algorithm engineer agents, algorithm training engineer agents, test engineer agents, deployment engineer agents, and algorithm application engineer agents.

[0035] Based on the A2A protocol, a structure format is defined to construct an agent call request structure, with fields including task number, target agent identifier, subtask index, call method, and call parameters;

[0036] After receiving the request structure, the agent generates a corresponding response structure, with fields including the response status code, response data, and exception information.

[0037] The invocation methods include synchronous invocation, asynchronous invocation, streaming invocation, and batch invocation; synchronous invocation is suitable for real-time feedback of task status, asynchronous invocation is suitable for non-blocking task scheduling, streaming invocation is suitable for task process tracking and feedback, and batch invocation is suitable for unified initiation and centralized return of multiple subtasks;

[0038] During protocol communication, call log information is recorded. The log content includes the call timestamp, agent interaction path, message body hash value, and status code information.

[0039] Based on the exception handling mechanism, the error field content in the response structure is identified. If a call exception is identified, the error retry module is triggered, and the task status record and error handling log are updated.

[0040] The A2A protocol communication layer supports scheduling agents to perform skill discovery operations, querying agent registration information and capability items through protocol broadcast requests, and updating the set of callable agents based on capability matching results.

[0041] Optionally, the specialized intelligent agent cluster layer specifically comprises:

[0042] Configure a role definition module, an LLM model, an MCP toolset, a knowledge base module, and a memory system in the professional intelligent agent cluster layer;

[0043] The role definition module defines the responsibilities, interaction interfaces, and capability tags of the data labeling agent, data analyst agent, algorithm engineer agent, algorithm training engineer agent, test engineer agent, deployment engineer agent, and algorithm application engineer agent.

[0044] After receiving the scheduling instruction, the data annotation agent calls the data annotation tool to complete the target annotation and label structure generation operation of image or video data; the data analyst agent performs feature extraction, distribution analysis, and data quality assessment on the structured labeled data, and outputs the analysis result structure; the algorithm engineer agent calls the algorithm component configuration template to complete the model structure setting, parameter initialization, and training process generation operation; the algorithm training engineer agent calls the MCP toolset to execute the model training process, and collects training logs and performance indicators to output a training report; the test engineer agent calls the test component based on the trained model to complete the accuracy verification, stability testing, and anomaly analysis tasks and outputs the test results; the deployment engineer agent calls the deployment configuration module to generate a model deployment plan, and completes the deployment path configuration and service interface release tasks; the algorithm application engineer agent receives the deployment model information, completes the inference call configuration and business interface joint debugging tasks, and collects application feedback information;

[0045] The LLM model is used to perform subtask understanding and collaboration plan generation operations, and output subtask objectives, processing requirements and interaction sequence information.

[0046] The MCP toolset calls stage tools to perform data cleaning, annotation generation, feature analysis, model building, training and optimization, testing and verification, and deployment and release operations.

[0047] The knowledge base module provides reference documents on visual domain knowledge and tool usage related to the task;

[0048] The system records task numbers, execution steps, interaction logs, and status feedback information through a memory system, and maintains the context of multi-round task execution and the response history of professional intelligent agents.

[0049] Optionally, the MCP toolset specifically includes:

[0050] The MCP toolset includes data processing tools, model building tools, training and tuning tools, testing and validation tools, and deployment and release tools.

[0051] The data processing tool component includes a data cleaning tool, a format conversion tool, and an anomaly detection tool, used to receive raw image or video data and output a standard format data structure; the model building tool component includes a model template library, a parameter configuration interface, and a structure visualization module, receiving model configuration parameters input by the algorithm engineer's agent and outputting the initial model definition structure; the training and optimization tool component includes a training task manager, a log collector, and a performance evaluation module, receiving the initial model and training data and outputting the trained model structure and performance report; the testing and verification tool component includes a test case generator, a stability verification module, and a result analysis module, receiving the trained model and test data and outputting test metric results; the deployment and release tool component includes a deployment configuration interface, a service packaging module, and an interface release manager, receiving the tested and approved model and deployment requirements and outputting the deployment package structure and API service documentation.

[0052] Each tool component interacts with the professional intelligent agent through a unified interface protocol, supports concurrent calls and task status reporting, and assists the scheduling intelligent agent in performing task status tracking and error handling operations by recording execution logs and providing error information feedback.

[0053] Optionally, the knowledge base module specifically includes:

[0054] Store the technical specifications, industry standards and best practices involved in the self-training of visual algorithms, and build structured knowledge entries, document knowledge sets, configuration templates and case libraries;

[0055] Establish a semantic index structure to construct vector representations for technical terms, task elements, and tool parameters, supporting fuzzy matching, semantic retrieval, and relevance ranking;

[0056] Based on the call requests submitted by the scheduling agent and the professional agent, the knowledge base module provides the domain knowledge content, algorithm theory information and tool usage reference documents required for the task;

[0057] It supports runtime knowledge update operations, collects new technology documents, training experience and optimization strategy information during task execution, and generates versioned knowledge entries;

[0058] Set up the metadata structure for knowledge entries, with fields including knowledge number, source type, applicable task type, version number, and access permission level, and record update time and maintenance records;

[0059] It provides a domain context information input interface for LLM models and professional intelligent agents, and converts retrieval results into structured knowledge fragments to input into the task reasoning process;

[0060] It supports the management of knowledge content in multiple formats, including text documents, configuration templates, operation examples and fault cases, and restricts the reading and submission permissions of different roles through access control mechanisms;

[0061] It records knowledge retrieval logs, access records, and call history information. Fields include task number, search keywords, returned item index, and call timestamp. It also supports building knowledge recommendation content based on historical records.

[0062] Optionally, if any step fails during task execution, the scheduling agent performs a rescheduling operation, specifically as follows:

[0063] The scheduling agent monitors the response results of professional agents, the status of communication calls, and the data content returned by the MCP toolset at each stage. It identifies task execution failures and records the location of failure nodes, failure type information, and failure time index.

[0064] Based on the failure type information, query the task dependency table, mark the status of affected subtask nodes, and update the execution validity identifier of the associated paths in the task execution graph;

[0065] Call the memory system to extract the historical input parameters, response content and context structure information corresponding to the failed subtasks, and construct the subtask context information structure.

[0066] Based on the recovery information structure and LLM model output, the subtask objectives, calling order and resource requirement parameters are regenerated, and the subtask dependency paths in the execution plan are corrected.

[0067] Query the agent's state table and professional capability context information, filter the set of available professional agents, and select the target agent resource;

[0068] Reconstruct the call request structure, update the task number, subtask index, call method and call parameter fields, and re-initiate the execution instruction of the failed subtask;

[0069] Record the rescheduling status information of the subtask in the task status log table, and mark the task stage status as in the retry process;

[0070] Record the entire process information of the rescheduling operation and construct failure scheduling log entries; the entire process information includes the failed task number, the location of the failed node, the classification of the failure reason, the rescheduling time, and the target intelligent agent identifier;

[0071] If the rescheduling operation is successful, the scheduling agent updates the task execution status to normal completion; if the number of rescheduling attempts exceeds the preset threshold, the scheduling agent terminates the execution process, generates an abnormal termination report, and marks the task status as execution failure.

[0072] Optionally, the self-training process based on task execution and multi-agent system response, and the output of results, specifically includes:

[0073] The scheduling agent initiates subtask call requests sequentially according to the execution plan and records the task path and response status;

[0074] Specialized intelligent agents execute sub-tasks and generate structured result data, which is then fed back to the scheduling intelligent agent through the A2A protocol communication layer;

[0075] The memory system records task number, task stage, call history, and status information;

[0076] The knowledge base module provides domain knowledge support during execution, providing structured knowledge fragments for specialized intelligent agents and LLM models;

[0077] The LLM model outputs subtask understanding results and collaboration plan information at key stages to help generate the final output.

[0078] The scheduling agent summarizes the output of each stage, generates the final result data and execution report of the self-training process, and records the task completion status.

[0079] The beneficial effects of this invention are:

[0080] This invention addresses the challenges of complex task understanding, fragmented execution processes, difficulties in module collaboration, and insufficient anomaly recovery in visual algorithm self-training tasks by constructing a multi-agent system architecture consisting of a user interaction layer, an intelligent scheduling layer, an A2A protocol communication layer, and a professional agent cluster layer. It employs an integrated LLM model for task understanding and decomposition, combined with professional capability context information constructed by the scheduling agent, to unify execution plan formulation and sub-task allocation strategies. The A2A protocol communication layer uses standardized calling methods to achieve asynchronous calls, streaming feedback, and skill discovery, resolving communication inconsistencies between multiple agents. The professional agent cluster layer configures a role definition module, an MCP toolset, a knowledge base module, and a memory system, clearly defining... The system defines the functional division of each type of intelligent agent, the task execution path, and the interface interaction specifications. Through the MCP toolset, it completes the execution of sub-tasks throughout the entire process, from data processing, model building, training and tuning, testing and verification to deployment and release. The knowledge base module provides structured knowledge support for the visual domain related to the task context, and the memory system maintains the task execution history and context state, effectively supporting task retries and dynamic scheduling. Ultimately, it realizes an intelligent process for visual algorithm tasks from natural language description to execution closed-loop management, improving the system's scheduling efficiency, task stability, and anomaly recovery capabilities in application scenarios with high task complexity, strong module collaboration, and real-time execution requirements. It also has the beneficial effects of strong scalability, high intelligent collaboration, and complete execution closed loop. Attached Figure Description

[0081] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0082] Figure 1 This is a flowchart of a self-training method for visual algorithms based on multi-agent collaborative optimization proposed in this invention;

[0083] Figure 2 This is a framework diagram of the multi-agent system proposed in this invention;

[0084] Figure 3 This is a schematic diagram illustrating the execution of a multi-agent cooperative task proposed in this invention;

[0085] Figure 4 This is a schematic diagram of inter-agent communication based on the A2A protocol proposed in this invention;

[0086] Figure 5 This is a diagram of the intelligent agent architecture proposed in this invention. Detailed Implementation

[0087] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0088] refer to Figure 1-2 A self-training method for visual algorithms based on multi-agent collaborative optimization includes the following steps:

[0089] S1. Construct a multi-agent system architecture; the multi-agent system includes a user interaction layer, an intelligent scheduling layer, an A2A protocol communication layer, and a professional agent cluster layer;

[0090] S2. In the user interaction layer, receive image or video data uploaded by the user, and parse and generate the user's task intent through the user interface that integrates the LLM model.

[0091] S3. The intelligent scheduling layer receives the user's task intent, constructs professional capability context information through the scheduling intelligent agent in the professional intelligent agent, and integrates the LLM model to complete task understanding and task decomposition, generates execution steps and dependencies, and forms an execution plan.

[0092] S4. Based on the execution plan, the scheduling agent calls upon specialized agents to execute corresponding sub-tasks;

[0093] S5. Establish a standardized communication mechanism in the A2A protocol communication layer to complete asynchronous calls, state updates, skill discovery and error handling between intelligent agents;

[0094] S6. In the professional intelligent agent cluster layer, the role definition module sets the professional division of labor, the LLM model executes the sub-task understanding and collaboration plan, the MCP toolset completes data processing and training, the knowledge base module provides domain knowledge support, and the memory system maintains the task context and interaction records.

[0095] S7. If any execution step fails during task execution, the scheduling agent performs a rescheduling operation; based on the task execution and multi-agent system response process, the self-training process is completed and the results are output.

[0096] This implementation method is based on a multi-agent collaborative optimization approach for visual algorithm self-training. It automates and automates the visual algorithm model training process by constructing a multi-agent system architecture comprising a user interaction layer, an intelligent scheduling layer, an A2A protocol communication layer, and a professional agent cluster layer. The user interaction layer integrates an LLM model to parse user natural language commands, identify task intent, and generate structured task requests. The intelligent scheduling layer, based on the LLM model, performs task understanding and decomposition, generates execution steps and dependencies, constructs an execution plan, and schedules professional agents to execute sub-tasks. The A2A protocol communication layer provides a standardized communication mechanism, supporting synchronous, asynchronous, streaming, and batch calls to ensure efficient and reliable information exchange between agents. The professional agent cluster layer defines roles for data annotation, data analysis, algorithm design, model training, testing and evaluation, deployment and implementation, and algorithm integration, collaboratively executing each stage of the visual task. Furthermore, the system introduces the MCP toolset to uniformly encapsulate tool components at each stage, enabling standardized invocation and execution status tracking of multiple tool types. Going further, the system integrates a structured knowledge base module, providing semantic retrieval and knowledge reasoning capabilities for specialized agents and LLM models, supporting runtime knowledge updates and historical experience accumulation. Simultaneously, a memory system records task context and interaction logs, supporting multi-round task state maintenance and agent response history synchronization. Based on these mechanisms, the system dynamically responds to the execution status and feedback information of each class during task execution. If any step fails, the scheduling agent regenerates the task path and resource configuration, executes sub-task rescheduling operations, and continuously completes the closed-loop self-training process. The final output includes the trained model, performance report, deployment package, and application feedback results, forming a complete delivery, significantly improving the efficiency, adaptability, and intelligence level of visual algorithm development.

[0097] In this embodiment, S2 specifically refers to:

[0098] The user interaction layer receives image or video data uploaded by users and performs format recognition and input registration on the received data.

[0099] The task description text associated with the input data is extracted through the user interface that integrates the LLM model; the task description text includes natural language text input by the user, instruction prompts, or content from uploaded task description files.

[0100] Task intent recognition is performed based on the task description text to generate task intent information. The task intent information includes the task objective, processing method, output requirements, and expected feedback form. The LLM model encodes the task description based on a pre-trained semantic encoding structure and a contextual attention mechanism, and identifies the task type label and processing requirements through the intent classification module. The intent classification module performs classification operations by constructing a task type vector space and introducing a multi-layer feedforward network. The intent label is fitted with the training set samples to obtain the classification weight.

[0101] The task intent information and input data are combined to construct a structured task request content; the structured task request content includes task number, data path, data type, task intent tag, output type, priority marker, and user identification information;

[0102] The structured content of the task request is submitted to the intelligent scheduling layer, and the task number, input data index, and task submission time information are recorded. The data index includes a file hash value, a task data mapping table, and a version number field, which are used to identify the consistency and traceability of the data in subsequent processes. The submission operation performs structured packaging through a standard data interface and calls the task registration interface of the scheduling agent.

[0103] refer to Figure 3 In this embodiment, the intelligent scheduling layer serves as the core decision-making center of the multi-agent system. It comprises scheduling agents forming a central scheduling module responsible for intelligent task parsing and resource scheduling operations. The scheduling agents deeply integrate an LLM model, possessing capabilities for natural language semantic parsing, task objective identification, and processing flow decomposition. They can receive user-submitted natural language task descriptions, obtain task intent, complexity, and processing objectives through semantic analysis, and construct execution plans based on the role information, capability tags, and available status of each specialized agent in the system. Specifically:

[0104] During the task reception phase, the scheduling agent constructs a set of professional capability context information, which includes the role definition, skill list, status identifier, and historical performance record of each professional agent. Based on the professional capability context information and the structured content of the task request, semantic parsing and task decomposition operations are performed through an integrated LLM model to generate a sub-task sequence, a dependency table, and a resource requirement list. The scheduling agent establishes a task execution graph according to the dependencies between sub-tasks and marks the priority, input type, and target output field of each task node.

[0105] The role definition module is configured according to the professional division of labor of intelligent agents in multi-agent systems, supporting professional roles such as data annotation, data analysis, algorithm development, model training, performance testing, deployment and implementation, and application integration, enabling intelligent agents to leverage their specific advantages in a particular technology stack;

[0106] When a user submits a task description such as "analyze video data and train an object detection model", the scheduling agent can identify the three stages involved in the task: data processing, object labeling, and model training. It can then construct corresponding sub-task nodes in the task graph and assign the corresponding professional agents to complete the distributed execution operations.

[0107] The scheduling agent structure comprises three functional units: an intelligent task analysis module, a dynamic agent discovery module, and an execution coordination module. The intelligent task analysis module integrates an LLM model, receives structured task requests, and performs task understanding, target identification, and execution step generation. The dynamic agent discovery module uses an active probing mechanism to call the A2A protocol communication interface to scan registered professional agent resource nodes, collecting their role identifiers, status codes, task response capabilities, and resource utilization information. The execution coordination module maintains a task status mapping table and an execution scheduling record table, performs task allocation, scheduling control, and concurrent status synchronization operations, and executes a failure rescheduling process when a task status is abnormal.

[0108] The scheduling agent possesses task lifecycle management capabilities, including subtask dependency handling, dynamic resource matching, failure retry mechanisms, and task termination policy control. Concurrent task scheduling adopts a non-blocking asynchronous call structure, allocates task resources to specialized agents based on an intelligent load balancing mechanism, and adjusts scheduling weights according to the agent's current number of tasks, processing latency, resource utilization, and historical success rate. During task execution, key performance indicators, including CPU utilization, memory usage, IO rate, and response latency, are collected in real time, and the task execution status view and system operation status monitoring graph are output through a visualization platform.

[0109] In terms of system scalability, the intelligent scheduling layer adopts a modular design structure, which supports new intelligent agents to automatically access the intelligent agent resource set through the registration protocol interface, and perform identity creation, capability tag binding and status monitoring initialization; the scheduling intelligent agent is compatible with both A2A protocol and traditional HTTP protocol, supports multi-protocol parallel scheduling mechanism, and improves cross-platform task scheduling capability and system compatibility.

[0110] By integrating the semantic parsing capabilities and standardized communication mechanisms of the LLM model into the intelligent scheduling layer, a complete task flow channel can be built between user natural language input and multi-agent distributed execution, improving the processing efficiency and execution stability of visual algorithm tasks. In the event of response anomalies or execution failures, the system can automatically perform strategy adjustments and task reallocation operations to ensure the continuous, stable, and efficient advancement of the self-training process.

[0111] In this embodiment, the LLM model serves as an inference engine, performing natural language understanding, logical reasoning, and task decision-making operations. It can perform semantic parsing, target recognition, and structured transformation operations on complex task instructions submitted by the user, generating structured task solutions that can be executed by the multi-agent system. The LLM model is integrated into the multi-agent system through a standard API interface, providing intelligent task analysis, task decomposition planning, and agent scheduling decision-making services, specifically:

[0112] The LLM model possesses the following key capabilities:

[0113] Task intent understanding capability, supporting semantic parsing and contextual understanding of user task descriptions, identifying task objectives, processing objects and implicit parameter requirements, and constructing task semantic representation vectors;

[0114] Intelligent decomposition and planning capabilities support parsing complex task structures into multi-level sub-task sequences, marking the dependency paths, execution order, and input-output relationships between tasks, and outputting sub-task sequences and dependency relationship tables;

[0115] The intelligent agent invocation capability supports analyzing the required professional capabilities based on the sub-task objectives, performing skill matching and resource filtering based on the registered intelligent agent capability context information, selecting the matching professional intelligent agent, and outputting the invocation request structure; the invocation request structure includes the skill request corresponding to the sub-task; the invocation process includes intelligent agent discovery, skill tag comparison, communication protocol adaptation, and intelligent agent status monitoring, and the invocation path follows the A2A protocol communication standard to ensure a unified invocation structure and complete status transmission;

[0116] Dynamic scheduling and optimization capabilities support dynamic adjustment of task allocation strategies based on the current resource status of the agent and task execution records, including scheduling priority optimization, task resource reallocation, and load weight correction.

[0117] It supports streaming response capabilities, enabling process feedback and intermediate result output during the reasoning process. It is suitable for multi-round task interaction, complex semantic reasoning, and step-by-step feedback output in long task chain response processes.

[0118] The LLM model is constructed as an asynchronous inference structure, supporting high-concurrency request processing. It has error recognition mechanisms, retry control mechanisms, and inference degradation mechanisms, which can maintain the stability and continuity of the system under high task load scenarios. The LLM model not only performs task analysis and policy formulation tasks, but also participates in the construction of agent collaboration paths and the closed-loop control of task flow in multi-agent systems. It outputs task-oriented execution graphs and inference explanations, enhancing the system's collaborative consistency and task controllability.

[0119] refer to Figure 4In this embodiment, the A2A protocol communication layer specifically comprises:

[0120] The A2A protocol communication layer establishes a self-defined communication standard protocol system to standardize message formats, interaction processes, and invocation methods. The communication protocol uses a unified structured data format to construct message content, with the default transmission structure being a field definition structure based on JSON format. The communication structure includes fields such as task number, source agent identifier, target agent identifier, subtask index, invocation method, invocation parameters, and status code, adapting to different invocation types and task scenarios.

[0121] The A2A protocol communication layer supports multiple communication modes, including synchronous call mode, asynchronous call mode, streaming call mode, and batch call mode. The synchronous call mode is suitable for task processes that require real-time response, the asynchronous call mode is suitable for non-blocking task distribution processes, the streaming call mode supports intermediate process feedback and real-time data interaction, and the batch call mode supports unified scheduling and centralized response output of multiple subtasks.

[0122] The communication layer constructs a standard semantic structure for inter-agent calls, has a built-in skill discovery mechanism, and supports scheduling agents to query the service capabilities, skill types, and interface specifications of the registered agent set through broadcast messages. It automatically constructs an agent capability index table and maintains a call path mapping table and a communication adaptation list.

[0123] The A2A protocol communication layer supports multiple communication protocol channels, including HTTP request-response channels and WebSocket streaming channels, allowing switching between appropriate communication methods at different stages of task execution; it also supports the transmission of intermediate results, task status updates, and error information feedback during task execution.

[0124] The communication layer has a built-in exception detection mechanism and retry control module. It performs error identification based on the communication status field, the error code field and the exception description field in the response structure. If a communication exception is detected, the retry mechanism is started and the error type, call path and number of retries are recorded. If the set retry limit is reached, the communication exception status is output and the subtask is marked as a call failure.

[0125] The communication layer constructs a communication log structure to record call timestamps, call paths, message body hash values, response status, and error types. The communication log supports task tracing, error backtracking, and performance analysis, and works with the scheduling agent to construct a task execution graph.

[0126] The A2A protocol communication layer unifies the intelligent agent calling process, interaction structure, and state synchronization method through a standardized communication mechanism, reducing the complexity of multi-agent integration, improving system scalability, inter-module interoperability, and operational stability, and supporting flexible collaboration and efficient interaction in various communication scenarios.

[0127] In this embodiment, the specialized intelligent agent cluster layer specifically comprises:

[0128] The professional intelligent agent cluster layer includes data labeling intelligent agents, data analyst intelligent agents, algorithm training engineer intelligent agents, test engineer intelligent agents, deployment engineer intelligent agents, and algorithm application engineer intelligent agents. Each type of intelligent agent has a corresponding set scope of responsibilities, capability tags, and interaction interface specifications, which support collaborative task execution and complementary process cooperation.

[0129] The data annotation agent is a data preparation expert for AI training pipelines, focusing on transforming raw multimedia data into structured training datasets; its professional fields cover computer vision, image processing, video analysis, and real-time streaming media processing; its core capabilities include object detection, image segmentation, video frame extraction, and automated annotation; it has the ability to integrate multiple advanced object detection algorithms and can handle multiple video formats and image types;

[0130] Data Analyst agents are experts in data science and machine learning data processing, focusing on dataset quality control, format conversion, and statistical analysis; their areas of expertise include data preprocessing, dataset management, quality assessment, and format standardization; their core capabilities cover data cleaning, format conversion, statistical analysis, and visualization reporting; they are proficient in multiple standard dataset formats and can handle stratified sampling and quality checks of large-scale datasets;

[0131] Algorithm engineers are experts in AI algorithm design and optimization, focusing on neural network architecture design, hyperparameter tuning, and algorithm innovation. Their expertise includes deep learning theory, model architecture design, and algorithm performance optimization. They are proficient in each type of neural network architecture (CNN, RNN, Transformer, etc.), optimization algorithms (Adam, SGD, etc.), and regularization techniques. They possess the ability to search for neural architectures and can design customized algorithms according to specific task requirements.

[0132] Algorithm Training Engineers are experts in deep learning model training, focusing on the training, optimization, and performance monitoring of neural network models. Their areas of expertise include deep learning algorithms, model architecture design, hyperparameter tuning, and training strategy optimization. Their core capabilities cover training various model architectures, distributed training, model fine-tuning, and performance monitoring.

[0133] Test Engineers are experts in AI model quality assurance, focusing on model performance evaluation, benchmarking, and quality control. Their areas of expertise include model evaluation metrics, benchmarking methods, A / B test design, and quality assurance processes. Their core competencies cover the calculation of various evaluation metrics, benchmark performance comparison, multi-model comparative analysis, and automated testing processes. They are proficient in statistical methods, performance analysis, and testing theory, and are able to design scientific evaluation schemes.

[0134] The Deployment Engineer AI agent is an expert in AI model production, focusing on model optimization, deployment, and hardware acceleration; its areas of expertise include model compression, hardware acceleration, edge computing, and production deployment; its core capabilities cover model quantization, inference optimization, edge device adaptation, and containerized deployment; it is proficient in multiple acceleration frameworks and familiar with optimization strategies for various hardware platforms;

[0135] Algorithm Application Engineers, as core experts in AI business implementation, focus on deeply integrating trained AI models with real-world business scenarios to achieve a complete value transformation from model training to business application. Their expertise covers business process analysis, system integration, solution architecture, and domain applications. Core capabilities include requirements analysis, autonomous back-end orchestration, business integration, and problem-solving. They possess cross-domain knowledge integration capabilities and experience in autonomous solution design, enabling them to independently design complete algorithm application processes based on task requirements and business scenario characteristics. Through continuous optimization of orchestration schemes and real-time adjustments, they transform technical capabilities into efficient business value.

[0136] The cluster layer design is based on a microservice architecture, where all intelligent agents are deployed independently as service units. An autonomous operation mechanism is maintained through a containerized runtime environment. There are no direct dependencies between each intelligent agent, and standardized asynchronous interactive operations are achieved through an A2A protocol communication layer. The distributed deployment structure has decoupling between modules, high fault tolerance, and dynamic scaling capabilities, and supports multi-task collaborative execution in concurrent task scenarios.

[0137] During the task response process, the professional intelligent agent cluster layer receives sub-task instructions according to the execution plan generated by the scheduling intelligent agent and completes the task operation of each stage according to the division of responsibilities. The intelligent agent receives structured task call requests through the A2A protocol, extracts field information such as task number, sub-task index, and call parameters, matches its own capability tags, executes the corresponding processing logic, and returns the structured response result to the scheduling intelligent agent.

[0138] The professional intelligent agent cluster layer supports a flexible expansion mechanism. After a new intelligent agent is deployed, the intelligent agent identifier, service description and capability items are automatically synchronized to the scheduling intelligent agent capability index table through the registration mechanism, thus building a complete system capability map and meeting the intelligent agent invocation and scheduling requirements of dynamically changing task needs.

[0139] By using a modular deployment approach at the cluster layer and setting up standard communication interfaces, a multi-agent collaborative architecture with high coordination, reliability, and maintainability is constructed to support the orderly execution of tasks at each stage of the visual algorithm self-training process.

[0140] In this embodiment, the MCP toolset specifically includes:

[0141] The MCP toolkit defines the bidirectional communication method between the model and external resources based on a standardized protocol framework. It uses a JSON-RPC structure to define the format of request messages, response messages, and notification messages, and supports various transmission mechanisms such as standard input / output and HTTP server event sending. The communication lifecycle is divided into an initialization phase, a running phase, and a closing phase to ensure that the connection establishment, state synchronization, and release processes comply with protocol requirements.

[0142] The MCP toolkit completes the standardized mapping between the MCP protocol structure and the A2A protocol structure through the protocol adaptation component, including message field conversion, status code mapping and exception information synchronization; and achieves non-intrusive access of the tool interface to the A2A communication architecture through an automatic packaging mechanism, maintaining the stability of the original functional structure of the tool.

[0143] The MCP toolkit sets up a unified tool invocation logic, which identifies the target path, skill name and invocation parameters of the invoked tool through the routing parsing module, and supports three invocation modes: synchronous invocation, asynchronous invocation and streaming invocation. The tool discovery mechanism supports dynamic tool registration and discovery, records tool identification, function description, skill interface and health status information, and provides the scheduling agent with the set of invoked tools and capability descriptions.

[0144] The MCP toolset adopts a modular design structure, including data processing tool components, model building tool components, training and optimization tool components, testing and verification tool components, deployment and release tool components, and application integration tool components. Each component connects to the professional intelligent agent cluster layer based on a unified protocol interface and participates in the task execution process through the A2A protocol scheduling mechanism.

[0145] The data processing tool component is used for cleaning, labeling, and structuring image and video data, including file labeling tools, real-time streaming labeling tools, and data format conversion tools; the model building tool component provides the ability to configure neural network structures, set hyperparameters, and generate model templates; the training and optimization tool component performs training task creation, training process control, and performance metric recording; the testing and validation tool component performs model performance evaluation, benchmark testing, and result report generation; the deployment and release tool component realizes model format conversion, inference optimization, and runtime environment configuration; and the application integration tool component supports algorithm flow orchestration, business interface integration, and application feedback collection.

[0146] Through the protocol unification and function integration of the MCP toolset, professional intelligent agent clusters can call tools at each stage during task execution to complete data processing, model building, training and verification, and deployment integration processes, ensuring the structural integrity of the multi-agent scheduling process and the reusability of tool-level task capabilities.

[0147] In this embodiment, the knowledge base module specifically comprises:

[0148] The knowledge base module establishes a structured knowledge storage system, which includes technical specifications, industry standards and best practices related to visual algorithm self-training. It constructs three types of resource structures: document knowledge set, configuration template set and case knowledge set. Each knowledge entry is accompanied by metadata structure, including knowledge number, source type, applicable task type, version number and access permission level, and records the entry update time and maintenance records.

[0149] The knowledge base module sets up a semantic indexing mechanism, constructs vector representations of task keywords, tool parameters, and professional terms based on knowledge content, realizes fuzzy matching and semantic relevance ranking capabilities, and supports multi-condition combination retrieval methods; combined with the LLM model inference process, the knowledge base module can provide a set of knowledge entries that match the task goal, tool type, and calling parameters, and outputs them as structured knowledge fragments as one of the input contexts of the LLM model;

[0150] The knowledge base module supports a runtime knowledge update mechanism. It collects newly generated training experience, optimization strategies and tool usage documents through the execution process of professional intelligent agent tasks, performs versioned storage and automatically generates new knowledge entries, updates the knowledge index structure and recommendation weights.

[0151] The knowledge base module supports the management of knowledge content in multiple formats, including text documents, configuration templates, operation examples and fault cases, and sets up an access control mechanism to limit the reading, editing and submission permissions of different professional intelligent agent roles, so as to ensure the security of knowledge and the controllability of the scope of knowledge use.

[0152] The knowledge base module records knowledge retrieval logs, access records, and call history information corresponding to task numbers and agent identifiers. It also constructs knowledge recommendation content based on historical tasks and call paths to improve the efficiency of knowledge matching and the accuracy of LLM model inference during task execution.

[0153] In this embodiment, if any step fails during task execution, the scheduling agent performs a rescheduling operation, specifically as follows:

[0154] The scheduling agent monitors the response results of professional agents, the status of communication calls, and the data content returned by the MCP toolset at each stage. It identifies task execution failures and records the location of failure nodes, failure type information, and failure time index.

[0155] Based on the failure type information, query the task dependency table, mark the status of affected subtask nodes, and update the execution validity identifier of the associated paths in the task execution graph;

[0156] Call the memory system to extract the historical input parameters, response content and context structure information corresponding to the failed subtasks, and construct the subtask context information structure.

[0157] Based on the recovery information structure and LLM model output, the subtask objectives, calling order and resource requirement parameters are regenerated, and the subtask dependency paths in the execution plan are corrected.

[0158] Query the agent's state table and professional capability context information, filter the set of available professional agents, and select the target agent resource;

[0159] Reconstruct the call request structure, update the task number, subtask index, call method and call parameter fields, and re-initiate the execution instructions of the failed subtask;

[0160] Record the rescheduling status information of the subtask in the task status log table, and mark the task stage status as in the retry process;

[0161] Record the entire process information of the rescheduling operation and construct failure scheduling log entries; the entire process information includes the failed task number, the location of the failed node, the classification of the failure reason, the rescheduling time, and the target intelligent agent identifier;

[0162] If the rescheduling operation is successful, the scheduling agent updates the task execution status to normal completion; if the number of rescheduling attempts exceeds the preset threshold, the scheduling agent terminates the execution process, generates an abnormal termination report, and marks the task status as execution failure.

[0163] In this embodiment, the self-training process based on task execution and multi-agent system response, and the output of results, specifically includes:

[0164] The scheduling agent initiates subtask call requests sequentially according to the execution plan and records the task path and response status; the specialized agent executes the subtasks and generates structured result data, which is fed back to the scheduling agent through the A2A protocol communication layer;

[0165] The memory system records task number, task stage, call history, and status information. The system constructs a multi-layered structure, including short-term session memory, long-term task memory, and system status memory, which respectively manage current interaction parameters, historical task data, and system running status content, supporting information recording and retrieval at different granularities. During task execution, the system automatically updates the memory content, structurally writing the input and output content of each round of interaction into the corresponding memory unit, and associating it with interaction time, agent identifier, and task stage information to achieve complete maintenance of the task context. A least recently used caching strategy and an importance scoring mechanism are introduced to periodically clean up low-frequency access and low-value memories. The system dynamically optimizes storage resource allocation; it ensures the recovery of historical memory content after system restart through a persistence mechanism, maintaining cross-task context continuity; in multi-agent collaboration scenarios, the memory system integrates with the A2A protocol communication layer to achieve synchronous sharing of context information among agents, supporting unified state and consistent parameters in sub-task stages; it supports actively pushing the memory structure corresponding to the current task number to the target agent, enhancing the accuracy and consistency of task response; and it maintains task interaction logs, recording task number, round number, input parameters, response content, and call path, providing data support for subsequent task dependency analysis and strategy recommendation.

[0166] The knowledge base module provides domain knowledge support during execution, providing structured knowledge fragments for professional agents and LLM models; the LLM model outputs subtask understanding results and collaboration plan information at key stages to assist in generating the final output content; the scheduling agent summarizes the output of each stage, generates the final result data and execution report of the self-training process, and records the task completion status.

[0167] Example 1:

[0168] To verify the feasibility of this invention in practice, it was applied to the scenario of "training a detection model for four types of pollution in rivers and lakes." The following example illustrates the collaborative training process of a multi-agent system in this scenario. (Reference) Figure 5 .

[0169] (I) Task Understanding and Intelligent Decomposition

[0170] When a user inputs the natural language command "Help me train a model for the four disordered phenomena of rivers and lakes", the system first enters the task understanding and intelligent decomposition stage. The scheduling agent first receives the user's command, performs task analysis through LLM, and identifies that this is an AI model training task, specifically involving the training requirements of a "four disordered phenomena of rivers and lakes" detection model. At the same time, it determines that the task requires multi-agent collaboration, involving multiple stages such as data annotation, data processing, and model training. Based on the system's registered list of professional agent capabilities, it matches the most suitable combination of collaborative agents.

[0171] (ii) Data preparation led by data labeling agents

[0172] Following LLM analysis, the scheduling agent formulates a detailed execution plan, breaking down complex tasks into ordered execution steps according to the A2A protocol standard. First, the data labeling agent is activated to prepare data. The scheduling agent first identifies image or video data related to "river and lake pollution" and sends a skill call request to the data labeling agent via the A2A protocol. Upon receiving the scheduling instruction, the data labeling agent begins executing the image or video labeling task, utilizing its integrated LLM model for image understanding and object detection labeling to accurately label the image or video data related to "river and lake pollution," providing a high-quality labeled dataset for subsequent model training. During subsequent training and optimization, if the scheduling agent determines that the model performance is substandard (e.g., low accuracy due to insufficient data quality), it may re-coordinate with the data labeling agent to perform data augmentation or supplementary labeling to improve the quality and diversity of the dataset. This cycle continues until the data quality meets the training requirements.

[0173] (III) Quality Assurance of Data Analyst Intelligent Agents

[0174] After the annotation task is completed, the scheduling agent immediately coordinates with the data analyst agent to perform data preprocessing based on the execution feedback. The data analyst agent is responsible for tasks such as dataset format conversion, dataset quality assessment, and partitioning. The annotated data is standardized to ensure it meets training requirements. Simultaneously, dataset quality is assessed, checking key indicators such as data distribution balance and annotation accuracy, providing a high-quality data foundation for model training. This stage ensures the standardization and reliability of data processing, laying a solid data foundation for subsequent training stages. If the testing agent reports unsatisfactory model performance, the scheduling agent re-invokes the data analyst agent to perform dataset augmentation, re-partitioning, or quality improvement to enhance data distribution and feature representation. This optimization loop continues until the dataset quality meets training requirements.

[0175] (iv) Algorithm design of the intelligent agent by the algorithm engineer

[0176] After data quality assurance is completed, the scheduling agent activates the algorithm engineer agent to perform algorithm design and model architecture planning. The algorithm engineer agent first selects the type of deep learning model based on task requirements and dataset characteristics, evaluates the performance trade-offs of various mainstream object detection algorithms, and chooses the most suitable model architecture for the real-time and accuracy requirements of the "river, lake, and four types of pollution" detection task. If the testing agent reports that the model performance does not meet the standards, the scheduling agent may reactivate the algorithm engineer agent to optimize the algorithm architecture to improve the model's detection accuracy and efficiency. This cycle continues until the algorithm design meets the performance requirements.

[0177] (v) Model training execution of AI training engineer's intelligent agent

[0178] After data preparation is complete, the scheduling agent initiates the core model training process. It sends training task creation and execution instructions to the AI ​​training engineer agent. The training engineer agent is responsible for creating the training task, configuring training parameters, and starting the model training process. During this stage, the system enters continuous monitoring mode. During training, the scheduling agent continuously monitors the training status via the A2A protocol, periodically obtaining performance metrics such as training progress, loss value, and accuracy. When the model performance is unsatisfactory (e.g., accuracy below a threshold, overfitting), the scheduling agent reactivates the AI ​​training engineer agent, which implements optimization strategies, including dynamic parameter adjustment, training strategy optimization, and iterative optimization loops. Based on training feedback, the scheduling agent may re-invoke the data analyst agent for dataset augmentation or re-partitioning, and adjust hyperparameters such as learning rate and batch size through the AI ​​training engineer agent, forming a closed-loop optimization process of "training → evaluation → adjustment → retraining." This adaptive training strategy ensures continuous performance optimization of the model on the "river and lake pollution" detection task.

[0179] (vi) Performance verification of the test engineer's intelligent agent

[0180] Triggered periodically after training or during training, the scheduling agent activates the test engineer agent to conduct a comprehensive model performance evaluation. The test agent performs systematic performance testing on the trained model, including calculating key metrics such as accuracy, recall, and F1 score, as well as comparing it with other benchmark models. It also generates a detailed test report, including performance analysis of the model in different scenarios and identification of potential problems. After testing, the test agent returns the evaluation results to the scheduling agent in the A2A protocol standard format. The LLM model integrated into the scheduling agent first presents the evaluation report to the user using natural language, and then performs in-depth analysis of the test results by reviewing performance benchmark data and historical training experience in the system knowledge base, or by optimizing based on user feedback. The system automatically determines whether the model performance meets the standards and whether there are key issues that need optimization. If the analysis results show that the model accuracy is insufficient, there is overfitting, or the model performs poorly in specific scenarios, the scheduling agent will automatically trigger corresponding optimization measures, such as re-calling the data analyst agent to adjust and enhance the dataset, or reactivating the training engineer agent to retrain the model. This cycle continues until the model performance reaches a satisfactory standard, forming a closed-loop performance optimization process. This ensures the actual usability of the model in the "river and lake pollution" detection task and provides reliable quality assurance for subsequent deployment and application.

[0181] (vii) Optimization preparation for deploying engineer intelligent agents

[0182] After the model evaluation is passed, the scheduling agent activates the deployment engineer agent to optimize the model and prepare for deployment based on deployment requirements. The deployment agent performs format conversion and hardware acceleration optimization on the model, including converting it to a format suitable for edge device deployment, performing GPU acceleration optimization, and adapting it for specific hardware platforms. This stage ensures that the model can run efficiently in the actual deployment environment, meeting the performance requirements of real-time detection and preparing the technical foundation for the practical application of the "River and Lake Pollution Prevention and Control" monitoring system.

[0183] (viii) Business integration of the intelligent agent by the algorithm application engineer

[0184] As a crucial link in the multi-agent collaborative process, the Algorithm Application Engineer agent bears the core responsibility of deeply integrating the trained AI model with real-world business scenarios. This is primarily achieved through autonomous post-link orchestration, realizing a complete value transformation from model training to business application. Based on task requirements and business scenario characteristics, the Algorithm Application Engineer agent autonomously designs a complete application process for the "Four Pollutions in Rivers and Lakes" detection algorithm. Autonomous orchestration capability is a core feature of the Algorithm Application Engineer agent. It can automatically adjust algorithm parameters and business rules according to different application scenarios, continuously optimize orchestration schemes based on operational data, and make real-time adjustments to respond to changes in business needs. Through autonomous post-link orchestration, the Algorithm Application Engineer agent ensures that the AI ​​algorithm can truly be implemented to solve real-world business problems, transforming the trained model into a usable business solution. This stage not only completes the technical deployment of the algorithm but also maximizes business value, providing river and lake management departments with efficient and reliable intelligent monitoring capabilities for the "Four Pollutions in Rivers and Lakes," achieving a complete closed loop from model training to business application.

[0185] (ix) Communication protocols and monitoring mechanisms between intelligent agents

[0186] Throughout the collaboration process, each specialized agent strictly adheres to the A2A protocol standard, employing a unidirectional communication architecture to ensure system controllability and security. All agent interactions utilize the standardized A2AMessage format, including fields such as message type, source and target agents, data payload, and association ID. The scheduling agent continuously monitors the state of each agent, acquiring real-time status information. Each agent reports its own status to the scheduling agent in real time, initiating health checks and fault tolerance mechanisms upon detecting anomalies to ensure the accuracy of scheduling decisions. Asynchronous streaming responses and WebSocket real-time communication are also supported, allowing users to observe the execution status of the entire training process in real time, including the current step, the state of each agent, training progress, and performance metrics.

[0187] (x) Training completion and result delivery

[0188] After the multi-agent collaboration process is completed, the scheduling agent integrates the execution results of each agent: collecting training logs, model files, performance reports, deployment packages, integration solutions, etc., to form a complete deliverable; conducting final quality verification to ensure the model meets the expected performance indicators; delivering the training results to the user and providing subsequent optimization suggestions. Finally, the scheduling agent coordinates the deployment engineer agent to optimize and convert the model's format, and delivers the trained model, performance report, and other complete results to the user. This multi-agent collaboration mode based on the A2A protocol realizes the automated decomposition and execution of the training task of the "Four Pollutions in Rivers and Lakes" environmental monitoring target detection model. Through the LLM-driven intelligent scheduling and real-time monitoring mechanism, it completes the identification of multiple categories of illegal behaviors involved in this task (including four typical environmental violations: illegal occupation of river and lake water areas, illegal mining of river and lake sand and gravel, illegal dumping of river and lake banks, and illegal construction in river and lake water areas), ensuring the efficiency and reliability of the "Four Pollutions in Rivers and Lakes" model training process, and providing a complete end-to-end AI solution for the field of environmental monitoring.

[0189] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A self-training method for visual algorithms based on multi-agent cooperative optimization, characterized in that, Includes the following steps: S1. Construct a multi-agent system architecture; the multi-agent system includes a user interaction layer, an intelligent scheduling layer, an A2A protocol communication layer, and a professional agent cluster layer; S2. In the user interaction layer, receive image or video data uploaded by the user, and parse and generate the user's task intent through the user interface that integrates the LLM model. S3. The intelligent scheduling layer receives the user's task intent, constructs professional capability context information through the scheduling intelligent agent in the professional intelligent agent, and integrates the LLM model to complete task understanding and task decomposition, generates execution steps and dependencies, and forms an execution plan. S4. Based on the execution plan, the scheduling agent calls upon specialized agents to execute corresponding sub-tasks; S5. Establish a standardized communication mechanism in the A2A protocol communication layer to complete asynchronous calls, state updates, skill discovery and error handling between intelligent agents; S6. In the professional intelligent agent cluster layer, the role definition module sets the professional division of labor, the LLM model executes the sub-task understanding and collaboration plan, the MCP toolset completes data processing and training, the knowledge base module provides domain knowledge support, and the memory system maintains the task context and interaction records. S7. If any execution step fails during task execution, the scheduling agent performs a rescheduling operation; based on the task execution and multi-agent system response process, the self-training process is completed and the results are output. The A2A protocol communication layer is specifically as follows: A standardized communication mechanism is established at the A2A protocol communication layer to support message exchange and command transmission between scheduling agents and specialized agents; the specialized agents include data labeling agents, data analyst agents, algorithm engineer agents, algorithm training engineer agents, test engineer agents, deployment engineer agents, and algorithm application engineer agents. Based on the A2A protocol, a structure format is defined to construct an agent call request structure, with fields including task number, target agent identifier, subtask index, call method, and call parameters; After receiving the request structure, the agent generates a corresponding response structure, with fields including the response status code, response data, and exception information. Invocation methods include synchronous invocation, asynchronous invocation, streaming invocation, and batch invocation; Synchronous calls are suitable for real-time feedback of task status, asynchronous calls are suitable for non-blocking task scheduling, streaming calls are suitable for task process tracking and feedback, and batch calls are suitable for unified initiation and centralized return of multiple subtasks. During protocol communication, call log information is recorded. The log content includes the call timestamp, agent interaction path, message body hash value, and status code information. Based on the exception handling mechanism, the error field content in the response structure is identified. If a call exception is identified, the error retry module is triggered, and the task status record and error handling log are updated. The A2A protocol communication layer supports scheduling agents to perform skill discovery operations, querying agent registration information and capability items through protocol broadcast requests, and updating the set of callable agents based on capability matching results; The specialized intelligent agent cluster layer is specifically as follows: Configure a role definition module, an LLM model, an MCP toolset, a knowledge base module, and a memory system in the professional intelligent agent cluster layer; The role definition module defines the responsibilities, interaction interfaces, and capability tags of the data labeling agent, data analyst agent, algorithm engineer agent, algorithm training engineer agent, test engineer agent, deployment engineer agent, and algorithm application engineer agent. After receiving the scheduling instruction, the data annotation agent calls the data annotation tool to complete the target annotation and label structure generation operation of the image or video data; the data analyst agent performs feature extraction, distribution analysis and data quality assessment on the structured labeled data, and outputs the analysis result structure; the algorithm engineer agent calls the algorithm component configuration template to complete the model structure setting, parameter initialization and training process generation operation; the algorithm training engineer agent calls the MCP toolset to execute the model training process, and collects training logs and performance indicators to output a training report; The test engineer agent calls the test components based on the trained model to complete accuracy verification, stability testing and anomaly analysis tasks and output test results; the deployment engineer agent calls the deployment configuration module to generate a model deployment plan and completes the deployment path configuration and service interface release tasks; the algorithm application engineer agent receives the deployment model information, completes the inference call configuration and business interface joint debugging tasks, and collects application feedback information. The LLM model is used to perform subtask understanding and collaboration plan generation operations, and output subtask objectives, processing requirements and interaction sequence information. The MCP toolset calls stage tools to perform data cleaning, annotation generation, feature analysis, model building, training and optimization, testing and verification, and deployment and release operations. The knowledge base module provides reference documents on visual domain knowledge and tool usage related to the task; The system records task numbers, execution steps, interaction logs, and status feedback information through a memory system, and maintains the context of multi-round task execution and the response history of professional intelligent agents. If any execution step fails during task execution, the scheduling agent performs a rescheduling operation, specifically as follows: The scheduling agent monitors the response results of professional agents, the status of communication calls, and the data content returned by the MCP toolset at each stage. It identifies task execution failures and records the location of failure nodes, failure type information, and failure time index. Based on the failure type information, query the task dependency table, mark the status of affected subtask nodes, and update the execution validity identifier of the associated paths in the task execution graph; Call the memory system to extract the historical input parameters, response content and context structure information corresponding to the failed subtasks, and construct the subtask context information structure. Based on the recovery information structure and LLM model output, the subtask objectives, calling order and resource requirement parameters are regenerated, and the subtask dependency paths in the execution plan are corrected. Query the agent's state table and professional capability context information, filter the set of available professional agents, and select the target agent resource; Reconstruct the call request structure, update the task number, subtask index, call method and call parameter fields, and re-initiate the execution instructions of the failed subtask; Record the rescheduling status information of the subtask in the task status log table, and mark the task stage status as in the retry process; Record the entire process information of the rescheduling operation and construct failure scheduling log entries; the entire process information includes the failed task number, the location of the failed node, the classification of the failure reason, the rescheduling time, and the target intelligent agent identifier; If the rescheduling operation is successful, the scheduling agent updates the task execution status to normal completion. If the number of rescheduling attempts exceeds a preset threshold, the scheduling agent will terminate the execution process, generate an abnormal termination report, and mark the task status as execution failure.

2. The self-training method for visual algorithms based on multi-agent cooperative optimization according to claim 1, characterized in that, Specifically, S2 is: The user interaction layer receives image or video data uploaded by users and performs format recognition and input registration on the received data. The task description text associated with the input data is extracted through the user interface that integrates the LLM model; Based on the task description text, perform task intent recognition to generate task intent information; then construct the task intent information and input data into structured content for the task request. The structured content of the task request is submitted to the intelligent scheduling layer, and the task number, input data index, and task submission time information are recorded.

3. The self-training method for visual algorithms based on multi-agent cooperative optimization according to claim 1, characterized in that, The intelligent scheduling layer includes: The scheduling agent receives structured content of task requests, establishes a task number index table, and registers task priority, task type, and input data source. The scheduling agent summarizes historical task records, professional agent role configurations, and knowledge base capability entries to generate professional capability context information. By integrating an LLM model, the structured content of task requests is analyzed to understand the task objectives, processing objects, and output format information. Based on task understanding, the task is decomposed by integrating an LLM model to generate a sequence of subtasks, a dependency table, and a resource requirement list, which constitutes the initial draft of the execution plan. The scheduling agent performs structural verification and resource matching on the initial draft of the execution plan, marks the executable status, and generates the execution plan.

4. The self-training method for visual algorithms based on multi-agent cooperative optimization according to claim 1, characterized in that, The LLM model is specifically as follows: The LLM model serves as an inference engine, receiving structured content from task requests. Based on the structured content of the task request, the task understanding is performed, and based on the task description text, the task objectives, processing objects and output requirements are identified to form semantic understanding results. Based on the semantic understanding results, intelligent task decomposition is performed to generate a sub-task sequence, a dependency table, and a resource requirement list; Based on the sub-task sequence, the agent invocation strategy is analyzed to identify the required professional capabilities and select a matching agent combination to construct a multi-agent collaborative process. The LLM model is suitable for multi-agent cooperative optimization. Dynamic task scheduling is adjusted based on agent state and resource availability, task allocation scheme is updated, and optimization feedback information is provided.

5. The self-training method for visual algorithms based on multi-agent cooperative optimization according to claim 1, characterized in that, The MCP toolset specifically includes: The MCP toolset includes data processing tools, model building tools, training and tuning tools, testing and validation tools, and deployment and release tools. The data processing tool component includes a data cleaning tool, a format conversion tool, and an anomaly detection tool, used to receive raw image or video data and output a standard format data structure; the model building tool component includes a model template library, a parameter configuration interface, and a structure visualization module, receiving model configuration parameters input by the algorithm engineer's agent and outputting the initial model definition structure; the training and optimization tool component includes a training task manager, a log collector, and a performance evaluation module, receiving the initial model and training data and outputting the trained model structure and performance report; the testing and verification tool component includes a test case generator, a stability verification module, and a result analysis module, receiving the trained model and test data and outputting test metric results; the deployment and release tool component includes a deployment configuration interface, a service packaging module, and an interface release manager, receiving the tested and approved model and deployment requirements and outputting the deployment package structure and API service documentation. Each tool component interacts with the professional intelligent agent through a unified interface protocol, supports concurrent calls and task status reporting, and assists the scheduling intelligent agent in performing task status tracking and error handling operations by recording execution logs and providing error information feedback.

6. The self-training method for visual algorithms based on multi-agent cooperative optimization according to claim 1, characterized in that, The knowledge base module specifically comprises: Store the technical specifications, industry standards and best practices involved in the self-training of visual algorithms, and build structured knowledge entries, document knowledge sets, configuration templates and case libraries; Establish a semantic index structure to construct vector representations for technical terms, task elements, and tool parameters, supporting fuzzy matching, semantic retrieval, and relevance ranking; Based on the call requests submitted by the scheduling agent and the professional agent, the knowledge base module provides the domain knowledge content, algorithm theory information and tool usage reference documents required for the task; It supports runtime knowledge update operations, collects new technology documents, training experience and optimization strategy information during task execution, and generates versioned knowledge entries; Set up the metadata structure for knowledge entries, with fields including knowledge number, source type, applicable task type, version number, and access permission level, and record update time and maintenance records; It provides a domain context information input interface for LLM models and professional intelligent agents, and converts retrieval results into structured knowledge fragments to input into the task reasoning process; It supports the management of knowledge content in multiple formats, including text documents, configuration templates, operation examples and fault cases, and restricts the reading and submission permissions of different roles through access control mechanisms; It records knowledge retrieval logs, access records, and call history information. Fields include task number, search keywords, returned item index, and call timestamp. It also supports building knowledge recommendation content based on historical records.

7. The self-training method for visual algorithms based on multi-agent cooperative optimization according to claim 1, characterized in that, The process of completing the self-training procedure and outputting results based on task execution and multi-agent system response is as follows: The scheduling agent initiates subtask call requests sequentially according to the execution plan and records the task path and response status; Specialized intelligent agents execute sub-tasks and generate structured result data, which is then fed back to the scheduling intelligent agent through the A2A protocol communication layer; The memory system records task number, task stage, call history, and status information; The knowledge base module provides domain knowledge support during execution, providing structured knowledge fragments for specialized intelligent agents and LLM models; The LLM model outputs subtask understanding results and collaboration plan information at key stages to help generate the final output. The scheduling agent summarizes the output of each stage, generates the final result data and execution report of the self-training process, and records the task completion status.

Images