Visual problem processing method and device based on multiple agents, equipment and medium

Abstract

The invention discloses a multi-agent-based visual problem processing method and device, equipment and a medium, and relates to the technical field of computer vision, and the method comprises the steps: obtaining a target sample image and a visual problem description corresponding to the target sample image, and carrying out the analysis of the target sample image through a visual understanding agent, so as to obtain a corresponding visual analysis report; generating a problem processing framework corresponding to the visual problem description based on the visual analysis report; and retrieving target data corresponding to the target sample image in a preset knowledge base by using a knowledge retrieval agent, and generating a target solution corresponding to the visual problem description based on the target data and the problem processing framework. A knowledge retrieval agent is utilized to retrieve a solution, a technical report and code data corresponding to a visual problem in a knowledge base in a multi-source mode, a target solution is generated on the basis of integration of a problem processing framework, and the problems that expert knowledge depends on and the technical scheme is separated from executable codes are synchronously solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a multi-agent based visual problem processing method, device, equipment and medium. Background Art

[0002] Computer vision refers to the use of cameras and computers to identify, track, and measure objects, replacing the human eye. This is further processed by the computer to create images more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data.

[0003] Current computer vision systems typically require customized development by experts to solve computer vision problems. This approach suffers from an over-reliance on expert knowledge and a disconnect between expert solutions and executable code. Therefore, reducing reliance on expert knowledge during image processing and improving the compatibility between image processing solutions and executable code have become unresolved technical challenges. Summary of the Invention

[0004] In view of this, the present invention aims to provide a multi-agent-based visual problem processing method, apparatus, device, and medium. This method utilizes a knowledge retrieval agent to search for solutions, technical reports, and code data corresponding to visual problems from multiple sources within a knowledge base, and integrates these into a target solution based on a problem processing framework. This method simultaneously addresses the issues of expert knowledge dependency and the separation of technical solutions from executable code. The specific solution is as follows:

[0005] In a first aspect, the present application provides a multi-agent based visual problem processing method, comprising:

[0006] Obtain a target sample image and a visual problem description corresponding to the target sample image, and analyze the target sample image using a visual understanding agent to obtain a corresponding visual analysis report;

[0007] generating a problem processing framework corresponding to the visual problem description based on the visual analysis report; wherein the problem processing framework includes solution evaluation indicators and problem solving indicators corresponding to the visual problem description, and the solution evaluation indicators include a problem solving accuracy indicator, a processing speed indicator, and a computing resource consumption indicator;

[0008] A knowledge retrieval agent is used to retrieve target data corresponding to the target sample image in a preset knowledge base, and a target solution corresponding to the visual problem description is generated based on the target data and the problem processing framework; wherein the target data includes solutions, technical reports and code data corresponding to historical visual problem descriptions, and each agent communicates through the A2A protocol.

[0009] Optionally, the using a visual understanding agent to analyze the target sample image to obtain a corresponding visual analysis report includes:

[0010] Extracting multi-dimensional features of the target sample image using the visual understanding agent; wherein the multi-dimensional features include color distribution, texture pattern, edge features, and spatial relationships corresponding to the target sample image;

[0011] The recognition difficulty of the target sample image is quantified using the visual understanding agent and the preset knowledge base to obtain a corresponding quantification result, and the visual analysis report corresponding to the target sample image is generated based on the multi-dimensional features and the quantification result.

[0012] Optionally, generating a problem processing framework corresponding to the visual problem description based on the visual analysis report includes:

[0013] Analyze the visual problem description and split the visual problem description into different sub-problems according to the corresponding analysis results;

[0014] Determining the correlation relationships between the sub-problems, generating problem-solving strategies corresponding to the sub-problems based on the correlation relationships, and determining the solution evaluation indicators corresponding to the problem-solving strategies;

[0015] The problem processing framework corresponding to the visual problem description is generated based on the visual analysis report and each of the solution evaluation indicators.

[0016] Optionally, the preset knowledge base includes a solution library, an academic literature library, and an open source code library, wherein the solution library is used to store solutions corresponding to historical visual problem descriptions, the academic literature library is used to store technical reports corresponding to historical visual problems, and the open source code library is used to store code data;

[0017] Accordingly, the method of using a knowledge retrieval agent to retrieve target data corresponding to the target sample image in a preset knowledge base includes:

[0018] The target data corresponding to the target sample image is retrieved in parallel in the solution library, the academic literature library and the open source code library using a knowledge retrieval agent.

[0019] Optionally, generating a target solution corresponding to the visual problem description based on the target data and the problem processing framework includes:

[0020] Determining the technical relevance between each target data and the visual problem description and the performance index of each target data, and scoring each target data based on a multi-dimensional scoring method, the technical relevance, and the performance index to obtain a target score corresponding to each target data;

[0021] Screening out data to be used whose target scores are greater than a preset score threshold from the target data, sorting the data to be used based on the target scores, and generating a knowledge resource package using the sorted data to be used;

[0022] The target solution corresponding to the visual problem description is generated based on the knowledge resource package and the problem processing framework.

[0023] Optionally, generating the target solution corresponding to the visual problem description based on the knowledge resource package and the problem processing framework includes:

[0024] Aligning the knowledge resource package with the problem-solving framework to select a first sub-problem and a second sub-problem from each of the sub-problems; wherein the first sub-problem is a problem that can be directly processed by the target algorithm model in the problem-solving framework, and the second sub-problem is a problem that cannot be directly processed by the target algorithm model;

[0025] An initial solution corresponding to the second sub-problem is determined, and the initial solution is optimized based on a conflict resolution algorithm to obtain the target solution corresponding to the visual problem description.

[0026] Optionally, after generating a target solution corresponding to the visual problem description based on the target data and the problem processing framework, the method further includes:

[0027] Obtaining target feedback data corresponding to the target solution, parsing the target feedback data, and extracting solution defects of the target solution from the target feedback data;

[0028] Based on the defects of the solution, the model parameters and model structure of the target algorithm model are adjusted to obtain a corresponding adjusted model, so as to use the adjusted model to process the new image processing task.

[0029] In a second aspect, the present application provides a multi-agent-based visual problem processing device, comprising:

[0030] An image analysis module is used to obtain a target sample image and a visual problem description corresponding to the target sample image, and analyze the target sample image using a visual understanding agent to obtain a corresponding visual analysis report;

[0031] a framework generation module, configured to generate a problem processing framework corresponding to the visual problem description based on the visual analysis report; wherein the problem processing framework includes solution evaluation indicators and problem solving indicators corresponding to the visual problem description, and the solution evaluation indicators include a problem solving accuracy indicator, a processing speed indicator, and a computing resource consumption indicator;

[0032] A solution generation module is used to use a knowledge retrieval agent to retrieve the target data corresponding to the target sample image in a preset knowledge base, and generate a target solution corresponding to the visual problem description based on the target data and the problem processing framework; wherein the target data includes solutions, technical reports and code data corresponding to historical visual problem descriptions, and each agent communicates through the A2A protocol.

[0033] In a third aspect, the present application provides an electronic device, comprising:

[0034] Memory, used to store computer programs;

[0035] A processor is used to execute the computer program to implement the aforementioned multi-agent based visual problem processing method.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned multi-agent-based visual problem processing method.

[0037] This application first obtains a target sample image and a visual problem description corresponding to the target sample image, uses a visual understanding agent to analyze the target sample image to obtain a corresponding visual analysis report, and then generates a problem processing framework corresponding to the visual problem description based on the visual analysis report; wherein, the problem processing framework includes solution evaluation indicators and problem solving indicators corresponding to the visual problem description, and the solution evaluation indicators include problem solving accuracy indicators, processing speed indicators and computing resource consumption indicators. Finally, a knowledge retrieval agent is used to retrieve the target data corresponding to the target sample image in a preset knowledge base, and a target solution corresponding to the visual problem description is generated based on the target data and the problem processing framework; wherein, the target data includes solutions, technical reports and code data corresponding to historical visual problem descriptions, and each agent communicates with each other through the A2A protocol. It can be seen that this application uses knowledge retrieval agents to conduct multi-source retrieval of solutions, technical reports and code data corresponding to historical visual problems in a preset knowledge base, and integrates and generates target solutions based on the problem processing framework, thereby simultaneously solving the problems of excessive reliance on expert knowledge, insufficient reuse of historical knowledge, and separation of technical solutions and executable codes; by utilizing collaboration between multiple agents, the full process from understanding visual problems to generating executable solutions is automated, providing a systematic visual problem-solving framework and improving the efficiency of computer vision problem processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 This is a flow chart of a multi-agent based visual problem processing method disclosed in this application;

[0040] Figure 2 This is a flowchart of a multi-agent based visual problem processing method disclosed in this application;

[0041] Figure 3 This is a schematic diagram of the structure of a multi-agent based visual problem processing device disclosed in this application;

[0042] Figure 4 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] Currently, computer vision systems often rely too heavily on expert knowledge for problem solving, and their solutions are often separated from executable code. This application proposes a multi-agent-based visual problem solving method that utilizes a knowledge retrieval agent to retrieve solutions, technical reports, and code data corresponding to visual problems from multiple sources within a knowledge base. This method then integrates and generates target solutions based on a problem solving framework, simultaneously resolving the issues of reliance on expert knowledge and the separation of technical solutions from executable code.

[0045] See also Figure 1 As shown, the embodiment of the present invention discloses a multi-agent based visual problem processing method, comprising:

[0046] Step S11: Obtain a target sample image and a visual problem description corresponding to the target sample image, and use a visual understanding agent to analyze the target sample image to obtain a corresponding visual analysis report.

[0047] This embodiment discloses a multi-agent visual problem-solving method and a corresponding computer vision technology solution-assisted generation system. Five specialized agents collaborate to perform visual problem analysis, solution design, and optimization. The system integrates the Model Context Protocol (MCP) and A2A protocols to enable tool invocation and agent collaboration. It features full-process automation, knowledge accumulation, and continuous optimization capabilities, enabling efficient generation of practical computer vision technology solutions.

[0048] This system adopts a distributed architecture, and each agent communicates through a standardized API. The core components include:

[0049] Agent management service, used for agent lifecycle management and resource allocation, as well as agent discovery and communication based on the A2A (Agent-to-Agent, an inter-agent communication protocol) protocol.

[0050] The question representation engine is used to convert visual questions into standardized expressions, which includes multimodal feature extraction and question feature vectorization modules.

[0051] A knowledge base system that hierarchically stores historical solutions, visual feature templates, and execution models, and supports semantic retrieval and similarity matching.

[0052] The visual tool adaptation layer is used to connect various visual analysis tools through the MCP protocol and provide a unified interface to access computer vision libraries and models.

[0053] The solution evaluation engine is used to evaluate the feasibility of solutions through simulation tests and provide solution comparison and optimization suggestions.

[0054] In this embodiment, the process of using multiple agents to process visual problems is as follows: Figure 2 As shown, we first need to accept the visual problem and have a preliminary understanding of it, and then analyze the visual features of the sample image to generate a preliminary solution. Then we conduct multi-source knowledge retrieval, and integrate and optimize the solutions based on the corresponding retrieval results. Finally, we generate the target solution and continuously optimize the solution based on user feedback.

[0055] In this embodiment, the central control coordination agent receives the visual problem description and sample image provided by the user, and forwards the problem and image to the visual understanding agent for analysis. The visual understanding agent performs a multi-dimensional analysis of the image and generates a structured problem definition report (i.e., a visual analysis report), including key visual features, challenge points, and potential solution paths. Accordingly, the process of using the visual understanding agent to analyze the target sample image to obtain the corresponding visual analysis report can specifically include: using the visual understanding agent to extract the multi-dimensional features of the target sample image; wherein the multi-dimensional features include the color distribution, texture pattern, edge features, and spatial relationships corresponding to the target sample image; using the visual understanding agent and a preset knowledge base to quantify the recognition difficulty of the target sample image to obtain the corresponding quantitative results, and generating a visual analysis report corresponding to the target sample image based on the multi-dimensional features and the quantification results.

[0056] The process described above is as follows: The central control coordination agent first receives the user-submitted visual problem description and related image samples (i.e., target image samples) and uses multimodal parsing techniques to understand the core of the problem. The system then performs a preliminary classification of the problem, identifies key requirements and technical constraints, and establishes an initial framework for problem solving. The central control agent then packages the problem and image into a standardized data package and securely transmits it to the visual understanding agent via the A2A protocol. This step also includes establishing the problem-solving session context, laying the foundation for subsequent analysis and solution generation. Furthermore, the system pre-allocates computing resources based on the problem complexity to ensure efficient analysis. After receiving the data package, the visual understanding agent deploys a multi-level visual feature extraction algorithm to comprehensively analyze the image. The system constructs a complete visual feature map by parallel processing multiple visual dimensions, including color distribution, texture features, edge characteristics, geometry, and contextual relationships. The agent then leverages an expert knowledge base to identify key challenging areas and feature points in the image and quantify their technical difficulty. Based on the analysis results, the system generates a structured problem definition report, which includes visual feature vectors, technical challenge classifications, and potential solutions. This report is returned to the central control agent through a standardized interface, providing a scientific basis for solution generation.

[0057] It should be noted that the central control coordination agent, as the core of the system, is responsible for global task scheduling and process management, and realizes dynamic task orchestration in different scenarios based on the workflow engine. Its decision-making module adopts a hybrid architecture, combining a rule engine and neural network reasoning to ensure the stability and adaptability of process control. The agent maintains a complete solution state diagram, tracks the execution status and result quality of each task node in real time, and supports conditional branching and exception handling. In terms of technical implementation, the central control agent is built based on a large language model, and improves its task planning capabilities through fine-tuning and knowledge enhancement. At the same time, it has a built-in context management mechanism to ensure the consistency of long-process tasks. The agent establishes a secure communication channel with other agents through the A2A protocol, supports both synchronous and asynchronous interaction modes, and implements message priority management and task rollback mechanisms. The central control agent also integrates a result synthesis engine that can integrate multi-source data to generate coherent technical solution documents.

[0058] The visual understanding agent focuses on image feature extraction and analysis, employing a multi-model ensemble architecture to enhance understanding accuracy. Its core comprises a group of pre-trained foundational visual models, including CNN (Convolutional Neural Network), Transformer (a model architecture), and hybrid architecture models, providing specialized processing capabilities for different types of visual tasks. The agent implements a multi-level feature analysis pipeline, from pixel-level feature extraction to high-level semantic understanding, sequentially analyzing image color distribution, texture patterns, edge features, shape recognition, and spatial relationships. When addressing complex visual problems, the agent draws on a domain expert knowledge base for auxiliary analysis, enhancing its ability to recognize domain-specific visual features. The analysis results are integrated through a feature fusion module to generate a structured visual problem description, including problem type classification, feature importance ranking, and identification of technical difficulties. The visual understanding agent supports an incremental learning mechanism, extracting patterns from historical cases to improve future analysis efficiency.

[0059] By adopting the MCP protocol to connect external visual tools and services, and leveraging the A2A protocol to achieve dynamic discovery and efficient collaboration between intelligent agents, a complete knowledge retrieval-solution generation-verification optimization cycle was built; through central control coordination, the entire process from visual problem understanding to executable solution generation was automated.

[0060] Step S12: generating a problem processing framework corresponding to the visual problem description based on the visual analysis report; wherein the problem processing framework includes solution evaluation indicators and problem solving indicators corresponding to the visual problem description, and the solution evaluation indicators include problem solving accuracy indicators, processing speed indicators and computing resource consumption indicators.

[0061] In this embodiment, the control and coordination intelligent agent generates a preliminary solution framework based on the visual analysis report and defines solution evaluation criteria and success indicators; accordingly, the process of generating a problem processing framework corresponding to the visual problem description based on the visual analysis report can specifically include: analyzing the visual problem description, and splitting the visual problem description into different sub-problems according to the corresponding analysis results; determining the correlation between each sub-problem, generating a problem processing strategy corresponding to each sub-problem based on each correlation, and determining the solution evaluation indicators corresponding to each problem processing strategy; generating a problem processing framework corresponding to the visual problem description based on the visual analysis report and each solution evaluation indicator.

[0062] Specifically, based on the visual analysis report, the central control coordination agent invokes the decision-making and reasoning engine to construct a preliminary solution framework (i.e., problem-solving framework). The system first decomposes the problem into multiple technical submodules (i.e., different subproblems) and develops corresponding technical routes and processing strategies for each submodule. The agent also sets quantitative criteria for solution evaluation, including key indicators such as accuracy, recall, processing speed, and resource consumption. The solution framework also includes a prioritized list of technical implementations and a dependency diagram to ensure the logical flow of subsequent development. After completing the preliminary solution, the system generates a technical requirements specification to provide precise guidance for knowledge retrieval.

[0063] By splitting the visual problem description into different sub-problems and formulating corresponding technical routes and processing methods for each sub-problem, the visual problem can be solved modularly, which can improve the intelligent agent's understanding of the visual problem and thus ensure the reliability of the generated solution.

[0064] Step S13: Use a knowledge retrieval agent to retrieve target data corresponding to the target sample image in a preset knowledge base, and generate a target solution corresponding to the visual problem description based on the target data and the problem processing framework; wherein the target data includes solutions, technical reports and code data corresponding to historical visual problem descriptions, and each agent communicates through an A2A protocol.

[0065] The knowledge retrieval agent is built as a multi-source information acquisition and processing center, using a hierarchical retrieval strategy to improve retrieval efficiency and quality. In the internal knowledge base retrieval module, the agent uses a vector database to implement similarity searches of high-dimensional features and quickly locate relevant historical cases. Its academic literature retrieval module integrates multiple academic search engines and databases through an API (Application Programming Interface), supporting advanced filtering and semantic retrieval. The code resource retrieval module implements structured crawling and analysis of different code platforms, capable of understanding code functionality and evaluating its quality and applicability. The agent has a built-in multi-dimensional scoring engine that comprehensively considers content relevance, source authority, technology maturity, and community support to intelligently sort search results. The retrieval agent also has the ability to analyze technology trends, identifying the direction of technological development in specific fields and providing forward-looking references for solution generation.

[0066] The preset knowledge base in this embodiment includes a solution base, an academic literature base and an open source code base. The solution base is used to store solutions corresponding to historical visual problem descriptions, the academic literature base is used to store technical reports corresponding to historical visual problems, and the open source code base is used to store code data. Accordingly, the process of using a knowledge retrieval agent to retrieve target data corresponding to the target sample image in the preset knowledge base can specifically include: using a knowledge retrieval agent to search for target data corresponding to the target sample image in parallel in the solution base, the academic literature base and the open source code base. That is, the knowledge retrieval agent simultaneously searches at three levels: an internal solution base, an academic literature and technical report base (i.e., an academic literature base), and an open source code and model repository (i.e., an open source code base), and sorts the search results by relevance and feasibility.

[0067] Specifically, after receiving the technical requirements specification, the knowledge retrieval agent initiates a three-way parallel search strategy to comprehensively acquire relevant technical resources. Within the internal solution library, the system uses a semantic vector matching algorithm to search for similar historical cases and their solutions. Simultaneously, the agent accesses academic literature databases, extracting relevant research papers, technical reports, and conference minutes to identify the latest theoretical breakthroughs. A third search strategy targets open source code platforms and model repositories, obtaining directly applicable algorithm implementations and pre-trained models.

[0068] In this embodiment, the process of generating a target solution corresponding to a visual problem description based on target data and a problem processing framework may specifically include: determining the technical relevance between each target data and the visual problem description and the performance indicators of each target data, and scoring each target data based on a multi-dimensional scoring method, technical relevance and performance indicators to obtain a target score corresponding to each target data; screening out data to be used whose target score is greater than a preset score threshold from the target data, sorting the data to be used based on the target score, and generating a knowledge resource package using the sorted data to be used; generating a target solution corresponding to the visual problem description based on the knowledge resource package and the problem processing framework; that is, the retrieval agent applies a multi-dimensional scoring algorithm to all results, comprehensively considering technical relevance, implementation difficulty, performance indicators and community activity, to form a knowledge resource package with optimized sorting.

[0069] The above-mentioned process of generating a target solution corresponding to the visual problem description based on the knowledge resource package and the problem processing framework may specifically include: aligning the knowledge resource package with the problem processing framework so as to screen out the first subproblem and the second subproblem from each subproblem; wherein the first subproblem is a problem that can be directly processed by the target algorithm model in the problem-solving framework, and the second subproblem is a problem that cannot be directly processed by the target algorithm model; determining the initial solution corresponding to the second subproblem, and optimizing the initial solution based on the conflict resolution algorithm to obtain the target solution corresponding to the visual problem description; that is, the central control coordination agent integrates the retrieval results, optimizes the preliminary solution, and discusses and modifies the feasibility of the solution with the verification agent through the A2A protocol; the execution adaptation agent converts the optimized solution into specific technical implementation steps, and selects a suitable technology stack based on the latest open source model to generate an executable solution containing code examples, parameter configurations and evaluation methods.

[0070] Specifically, after receiving the knowledge resource package, the central control coordination agent initiates the solution integration and optimization process to improve the feasibility and efficiency of the preliminary solution. The system first maps and aligns the retrieved technical solution with the preliminary framework (i.e., the problem-solving framework), identifying components that can be directly adopted (i.e., components corresponding to the first sub-problem) and parts that require customized development (i.e., components corresponding to the second sub-problem). Through the A2A protocol, the central control agent and the solution verification agent establish a dynamic negotiation channel to conduct in-depth discussions and feasibility assessments on key technical points. During this process, the system applies a conflict resolution algorithm to balance the advantages and disadvantages of different technical routes and consider the constraints of the actual deployment environment. The optimized architecture takes into account both technological advancement and implementation feasibility, and is expressed as a detailed technical architecture design document.

[0071] The execution adaptation agent receives the optimized technical architecture design and converts it into a specific, executable technical implementation plan (i.e., the target solution). The agent first analyzes the current open source technology ecosystem and selects the most suitable basic framework and toolchain from the technology stack mapping library. Based on the template engine, the system generates code examples for the core algorithms, including data preprocessing procedures, model building code, and inference deployment scripts. At the same time, the agent generates detailed parameter configuration documentation covering model structure parameters, training hyperparameters, and optimization strategy recommendations. The execution plan also includes a complete deployment architecture diagram, a list of environmental dependencies, and performance evaluation methods to ensure that users can implement the plan directly. All documents are organized in a structured format for easy understanding and execution.

[0072] Among them, the execution adaptation agent mentioned above is responsible for converting abstract technical solutions into directly deployable implementation solutions, bridging the gap between theory and practice. The agent has a built-in technology stack mapping engine, maintains the latest computer vision technology ecosystem map, and can select the most appropriate technology combination based on task requirements and performance constraints. Its parameter configuration generator recommends high-performance initial parameters for different models and algorithms based on Bayesian optimization and historical experience, accelerating the tuning process. The code template system maintains a large library of domain-specific code snippets, which can automatically assemble and generate core algorithm implementations based on technical solutions, including data preprocessing, model definition, and inference deployment code. The agent also provides a complete environment configuration document generation function, which lists software and hardware dependencies and version requirements in detail to ensure consistency in the deployment environment. The execution plan contains a detailed test verification guide, specifies acceptance criteria and performance baselines, and helps users evaluate the implementation effect.

[0073] The solution verification agent focuses on technical feasibility assessment and risk prediction to ensure the actual value of the generated solution. The agent uses similarity analysis technology based on historical cases to extract empirical patterns from historical success and failure cases to predict the success probability of the current solution. Its performance and resource demand prediction module combines algorithm complexity analysis and empirical models to estimate the computing resources, memory usage and time cost required for solution implementation. The risk identification subsystem uses multi-angle analysis to pre-identify technical bottlenecks, dependency risks and scalability issues, and provides risk level assessment and mitigation suggestions. The solution verification agent supports automated test scenario generation and can build typical and edge test cases to verify the stability of the solution under different conditions. The agent also integrates sensitivity analysis tools to evaluate the robustness of the solution to input changes through parameter perturbation testing, and generates visual performance fluctuation charts to assist decision-making.

[0074] After receiving user feedback, the central control coordination agent collaborates with other agents to adjust the solution based on the feedback, forming an optimization cycle until the problem is solved. Accordingly, in this embodiment, after generating a target solution corresponding to the visual problem description based on the target data and the problem processing framework, the process also includes: obtaining target feedback data corresponding to the target solution; parsing the target feedback data; and extracting solution defects of the target solution from the target feedback data; and adjusting the model parameters and model structure of the target algorithm model based on the solution defects to obtain a corresponding adjusted model, so that the adjusted model can be used to handle new image processing tasks.

[0075] Specifically, the system receives feedback from users after implementation and initiates an intelligent solution optimization cycle to ensure that the final solution meets actual needs. The central control coordination agent analyzes user feedback, extracts key problem points and performance shortcomings, and establishes an optimization task list. Subsequently, the central control agent coordinates various professional agents to propose improvement plans for different problem areas, including model structure adjustments, feature engineering optimization, and parameter fine-tuning suggestions. The system supports multiple rounds of iterative optimization, each round generating incremental improvement plans and predicting performance improvement space. During this process, the knowledge retrieval agent may be reactivated to find the latest solutions to specific problems. The entire optimization process maintains complete version control, recording the effects of each adjustment and the basis for decision-making.

[0076] After completing solution optimization, the system enters the knowledge accumulation phase, systematically preserving the solution and providing a foundation for future problem solving. The intelligent agent first constructs a complete problem-solution knowledge graph, documenting problem characteristics, solution paths, key decision points, and performance indicators. The system then modularizes the solution, marking reusable components and extracting domain-independent solution patterns. Using knowledge distillation techniques, expert experience is transformed into structured rules, enhancing the intelligence of future solution generation. Simultaneously, the system updates the problem-solution mapping relationship library and optimizes the weight parameters of the similar problem retrieval algorithm. The knowledge accumulation process also includes solution labeling and multidimensional index construction, ensuring efficient retrieval and reuse of currently accumulated technical knowledge in the future.

[0077] It can be seen that this application uses knowledge retrieval agents to conduct multi-source retrieval of solutions, technical reports and code data corresponding to historical visual problems in a preset knowledge base, and integrates and generates target solutions based on the problem processing framework, thereby simultaneously solving the problems of excessive reliance on expert knowledge, insufficient reuse of historical knowledge, and separation of technical solutions and executable codes; by utilizing collaboration between multiple agents, the full process from understanding visual problems to generating executable solutions is automated, providing a systematic visual problem-solving framework and improving the efficiency of computer vision problem processing.

[0078] See also Figure 3 As shown, an embodiment of the present invention discloses a multi-agent based visual problem processing device, comprising:

[0079] An image analysis module 11 is configured to obtain a target sample image and a visual problem description corresponding to the target sample image, and analyze the target sample image using a visual understanding agent to obtain a corresponding visual analysis report;

[0080] A framework generation module 12 is configured to generate a problem processing framework corresponding to the visual problem description based on the visual analysis report; wherein the problem processing framework includes solution evaluation indicators and problem solving indicators corresponding to the visual problem description, and the solution evaluation indicators include a problem solving accuracy indicator, a processing speed indicator, and a computing resource consumption indicator;

[0081] The solution generation module 13 is used to use the knowledge retrieval agent to retrieve the target data corresponding to the target sample image in the preset knowledge base, and generate the target solution corresponding to the visual problem description based on the target data and the problem processing framework; wherein, the target data includes the solution, technical report and code data corresponding to the historical visual problem description, and each agent communicates through the A2A protocol.

[0082] It can be seen that this application uses knowledge retrieval agents to conduct multi-source retrieval of solutions, technical reports and code data corresponding to historical visual problems in a preset knowledge base, and integrates and generates target solutions based on the problem processing framework, thereby simultaneously solving the problems of excessive reliance on expert knowledge, insufficient reuse of historical knowledge, and separation of technical solutions and executable codes; by utilizing collaboration between multiple agents, the full process from understanding visual problems to generating executable solutions is automated, providing a systematic visual problem-solving framework and improving the efficiency of computer vision problem processing.

[0083] In some specific implementations, the image analysis module 11 may specifically include:

[0084] a feature extraction unit, configured to extract multi-dimensional features of the target sample image using the visual understanding agent; wherein the multi-dimensional features include color distribution, texture pattern, edge features, and spatial relationships corresponding to the target sample image;

[0085] An analysis report generation unit is used to quantify the recognition difficulty of the target sample image using the visual understanding agent and the preset knowledge base to obtain a corresponding quantitative result, and generate the visual analysis report corresponding to the target sample image based on the multi-dimensional features and the quantitative result.

[0086] In some specific implementations, the framework generation module 12 may specifically include:

[0087] a problem splitting unit, configured to analyze the visual problem description and split the visual problem description into different sub-problems according to corresponding analysis results;

[0088] a processing strategy generating unit, configured to determine the correlation between the sub-problems, generate a problem processing strategy corresponding to each sub-problem based on the correlation, and determine the solution evaluation index corresponding to each problem processing strategy;

[0089] A framework generating unit is configured to generate the problem processing framework corresponding to the visual problem description based on the visual analysis report and each of the solution evaluation indicators.

[0090] In some specific implementations, the solution generation module 13 may specifically include:

[0091] A data retrieval unit is used to use a knowledge retrieval agent to search the target data corresponding to the target sample image in parallel in the solution library, the academic literature library and the open source code library.

[0092] In some specific implementations, the solution generation module 13 may specifically include:

[0093] a scoring acquisition unit, configured to determine a technical relevance between each target data and the visual problem description and a performance indicator of each target data, and score each target data based on a multi-dimensional scoring method, the technical relevance, and the performance indicator to obtain a target score corresponding to each target data;

[0094] a resource package generating unit, configured to filter out data to be used whose target scores are greater than a preset score threshold from the target data, sort the data to be used based on the target scores, and generate a knowledge resource package using the sorted data to be used;

[0095] A solution generation submodule is used to generate the target solution corresponding to the visual problem description based on the knowledge resource package and the problem processing framework.

[0096] In some specific implementations, the solution generation submodule may specifically include:

[0097] a data alignment unit, configured to align the knowledge resource package with the problem-solving framework, so as to select a first sub-problem and a second sub-problem from each of the sub-problems; wherein the first sub-problem is a problem that can be directly processed by the target algorithm model in the problem-solving framework, and the second sub-problem is a problem that cannot be directly processed by the target algorithm model;

[0098] A solution generation unit is used to determine an initial solution corresponding to the second sub-problem, and optimize the initial solution based on a conflict resolution algorithm to obtain the target solution corresponding to the visual problem description.

[0099] In some specific implementations, the solution generation module 13 further includes:

[0100] a feedback acquisition unit, configured to acquire target feedback data corresponding to the target solution, parse the target feedback data, and extract solution defects of the target solution from the target feedback data;

[0101] The model adjustment unit is used to adjust the model parameters and model structure of the target algorithm model based on the defects of the solution to obtain a corresponding adjusted model, so as to use the adjusted model to process the new image processing task.

[0102] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0103] Figure 4 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the multi-agent-based visual problem processing method disclosed in any of the aforementioned embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0104] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0105] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0106] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs capable of implementing the multi-agent-based visual problem processing method executed by the electronic device 20 as disclosed in any of the aforementioned embodiments, the computer program 222 may further include computer programs capable of implementing other specific tasks.

[0107] Furthermore, this application discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned multi-agent visual problem processing method. The specific steps of this method can be found in the corresponding content disclosed in the aforementioned embodiments and will not be repeated here.

[0108] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0109] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0110] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0111] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0112] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A multi-agent based visual problem processing method, characterized in that: include: Obtain a target sample image and a visual problem description corresponding to the target sample image, and analyze the target sample image using a visual understanding agent to obtain a corresponding visual analysis report; generating a problem processing framework corresponding to the visual problem description based on the visual analysis report; wherein the problem processing framework includes solution evaluation indicators and problem solving indicators corresponding to the visual problem description, and the solution evaluation indicators include a problem solving accuracy indicator, a processing speed indicator, and a computing resource consumption indicator; A knowledge retrieval agent is used to retrieve target data corresponding to the target sample image in a preset knowledge base, and a target solution corresponding to the visual problem description is generated based on the target data and the problem processing framework; wherein the target data includes solutions, technical reports and code data corresponding to historical visual problem descriptions, and each agent communicates through the A2A protocol.

2. The multi-agent based visual problem processing method according to claim 1, characterized in that: The using of a visual understanding agent to analyze the target sample image to obtain a corresponding visual analysis report includes: Extracting multi-dimensional features of the target sample image using the visual understanding agent; wherein the multi-dimensional features include color distribution, texture pattern, edge features, and spatial relationships corresponding to the target sample image; The recognition difficulty of the target sample image is quantified using the visual understanding agent and the preset knowledge base to obtain a corresponding quantification result, and the visual analysis report corresponding to the target sample image is generated based on the multi-dimensional features and the quantification result.

3. The multi-agent based visual problem processing method according to claim 1, characterized in that: Generating a problem processing framework corresponding to the visual problem description based on the visual analysis report includes: Analyze the visual problem description and split the visual problem description into different sub-problems according to the corresponding analysis results; Determining the correlation relationships between the sub-problems, generating problem-solving strategies corresponding to the sub-problems based on the correlation relationships, and determining the solution evaluation indicators corresponding to the problem-solving strategies; The problem processing framework corresponding to the visual problem description is generated based on the visual analysis report and each of the solution evaluation indicators.

4. The multi-agent based visual problem processing method according to claim 1, characterized in that: The preset knowledge base includes a solution library, an academic literature library and an open source code library. The solution library is used to store solutions corresponding to historical visual problem descriptions, the academic literature library is used to store technical reports corresponding to historical visual problems, and the open source code library is used to store code data. Accordingly, the method of using a knowledge retrieval agent to retrieve target data corresponding to the target sample image in a preset knowledge base includes: The target data corresponding to the target sample image is retrieved in parallel in the solution library, the academic literature library and the open source code library using a knowledge retrieval agent.

5. The multi-agent based visual problem processing method according to claim 3, characterized in that: Generating a target solution corresponding to the visual problem description based on the target data and the problem processing framework includes: Determining the technical relevance between each target data and the visual problem description and the performance index of each target data, and scoring each target data based on a multi-dimensional scoring method, the technical relevance, and the performance index to obtain a target score corresponding to each target data; Screening out data to be used whose target scores are greater than a preset score threshold from the target data, sorting the data to be used based on the target scores, and generating a knowledge resource package using the sorted data to be used; The target solution corresponding to the visual problem description is generated based on the knowledge resource package and the problem processing framework.

6. The multi-agent based visual problem processing method according to claim 5, characterized in that: Generating the target solution corresponding to the visual problem description based on the knowledge resource package and the problem processing framework includes: Aligning the knowledge resource package with the problem-solving framework to select a first sub-problem and a second sub-problem from each of the sub-problems; wherein the first sub-problem is a problem that can be directly processed by the target algorithm model in the problem-solving framework, and the second sub-problem is a problem that cannot be directly processed by the target algorithm model; An initial solution corresponding to the second sub-problem is determined, and the initial solution is optimized based on a conflict resolution algorithm to obtain the target solution corresponding to the visual problem description.

7. The multi-agent based visual problem processing method according to claim 1, characterized in that: After generating a target solution corresponding to the visual problem description based on the target data and the problem processing framework, the method further includes: Obtaining target feedback data corresponding to the target solution, parsing the target feedback data, and extracting solution defects of the target solution from the target feedback data; Based on the defects of the solution, the model parameters and model structure of the target algorithm model are adjusted to obtain a corresponding adjusted model, so as to use the adjusted model to process the new image processing task.

8. A multi-agent based visual problem processing device, characterized in that: include: An image analysis module is used to obtain a target sample image and a visual problem description corresponding to the target sample image, and analyze the target sample image using a visual understanding agent to obtain a corresponding visual analysis report; a framework generation module, configured to generate a problem processing framework corresponding to the visual problem description based on the visual analysis report; wherein the problem processing framework includes solution evaluation indicators and problem solving indicators corresponding to the visual problem description, and the solution evaluation indicators include a problem solving accuracy indicator, a processing speed indicator, and a computing resource consumption indicator; A solution generation module is used to use a knowledge retrieval agent to retrieve the target data corresponding to the target sample image in a preset knowledge base, and generate a target solution corresponding to the visual problem description based on the target data and the problem processing framework; wherein the target data includes solutions, technical reports and code data corresponding to historical visual problem descriptions, and each agent communicates through the A2A protocol.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the multi-agent based visual problem processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store a computer program, which, when executed by a processor, implements the multi-agent based visual problem processing method according to any one of claims 1 to 7.

Images