Intelligent agent cooperation method based on multi-modal data fusion

Abstract

The invention discloses an agent cooperation method based on multi-modal data fusion, and belongs to the technical field of multi-modal data fusion, and the method comprises the steps of dynamically collecting multi-source heterogeneous data, automatically distributing agent roles based on reinforcement learning, and optimizing task execution efficiency. The method comprises the following steps: identifying data, classifying the data according to the characteristics of the data, and establishing an efficient data flow management system for each identified data type to control the speed and sequence of the data flowing into the system from different sources, define the environment of agent operation and create a detailed capability model for each agent; historical data and a simulation scene are used to train an agent, when a new task arrives, the system determines a most suitable agent role allocation scheme in real time by using a trained reinforcement learning algorithm according to a current environment state, and feeds back an actual result to the reinforcement learning module during the task execution period of the agent.

Description

Technical Field

[0001] The present invention belongs to the technical field of multimodal data fusion, and specifically refers to an intelligent agent collaboration method based on multimodal data fusion. Background Art

[0002] Agent collaboration methods based on multimodal data fusion aim to improve the efficiency of cooperation and task execution among agents by integrating information from different data sources. With the development of the Internet of Things and artificial intelligence technologies, this approach has become particularly important, especially when dealing with complex and dynamic environments. Multimodal data fusion overcomes the limitations of a single data source and enhances the system's perception and decision-making accuracy by combining complementary information provided by multiple perception modes.

[0003] However, the existing multimodal data fusion intelligent agent collaboration method still has certain defects. The existing multimodal data fusion intelligent agent collaboration method reinforcement learning driven intelligent agent role allocation requires real-time environment status evaluation and dynamic adjustment, which may lead to high computational overhead due to high model complexity, especially in large-scale tasks. The real-time performance is insufficient. Cross-modal fusion relies on modal reliability assessment. If data noise or outliers are not completely repaired, it may lead to deviations in fusion results or even amplify errors. The cross-modal attention mechanism needs to map different modalities to a unified space, but the semantic differences between modalities are large, which may lead to insufficient semantic alignment. The dynamic role allocation and task decomposition require frequent adjustments to the collaboration of intelligent agents. If the environmental state changes too quickly, the reinforcement learning strategy may lag in response due to slow convergence. Although the Bayesian reasoning framework integrates multimodal features, the decision-making process is complex and difficult to explain. The visual processing ability in multimodal tasks is weak, and cross-modal associations may not be able to make up for the inherent deficiencies of the visual model. Hierarchical task planning and edge computing rely on predefined scenarios. If an untrained extreme situation is encountered, the system may not be able to dynamically adjust the strategy, causing the task to fail. For this reason, an intelligent agent collaboration method based on multimodal data fusion is proposed. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent agent collaboration method based on multimodal data fusion to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an agent collaboration method based on multimodal data fusion, comprising the following steps:

[0006] S1. Dynamically collect multi-source heterogeneous data, automatically assign agent roles based on reinforcement learning, and optimize task execution efficiency;

[0007] S2. Use the GAN diffusion model to repair low-quality data, combining cross-modal correlation and dynamic normalization to achieve adaptive preprocessing and data quality improvement;

[0008] S3, extract features through modality-specific encoders and use cross-modal attention mechanisms to achieve semantic alignment and eliminate the semantic gap;

[0009] S4, early, late and hybrid fusion strategies, dynamically adjust fusion weights based on modal reliability to balance the contribution of multimodal information;

[0010] S5. Hierarchically plan tasks and decompose them into subtasks. Use resource scheduling and consensus algorithms to coordinate agent collaboration, resolve conflicts, and balance loads.

[0011] S6. Design reinforcement learning-driven communication protocols to optimize communication efficiency and security, and support privacy protection and fault tolerance mechanisms;

[0012] S7. Combine Bayesian reasoning with multimodal voting mechanisms to handle uncertainty, introduce human-in-the-loop review, and improve decision robustness;

[0013] S8, low-latency action execution based on edge computing, continuous optimization of system performance through real-time feedback and online learning;

[0014] S9. Build a self-evolutionary framework, use knowledge distillation and meta-learning to achieve cross-task migration, and improve the long-term adaptability of intelligent agents.

[0015] Among them, in S1, the system first scans and identifies all available data sources, and classifies them according to the characteristics of the data. For each identified data type, the corresponding adapter is dynamically loaded to establish an efficient data flow management system to control the speed and order of data flowing into the system from different sources, define the environment in which the agent operates, create a detailed capability model for each agent, and train the agent using historical data and simulation scenarios. When a new task arrives, the system uses the trained reinforcement learning algorithm to determine the most appropriate agent role allocation plan in real time based on the current environment state, and feeds back the actual results to the reinforcement learning module during the agent's task execution.

[0016] Among them, S2 performs a preliminary quality assessment on the collected multi-source heterogeneous data, identifies noise, missing values, and outliers in the data, selects an appropriate generative adversarial network (GAN) model for data repair based on the data type, applies the selected GAN model to predict and fill in the missing parts in the data, uses the GAN model to remove unnecessary interference signals for noisy data sets, detects and corrects outliers in the data, extracts features from different types of input data, and attempts to find the correlation between them, uses appropriate algorithms for cross-modal information fusion, implements a dynamic standardization process, establishes a continuous monitoring and feedback system to track the data processing effect, and adjusts the parameter settings in the previous steps accordingly. The implementation formula is:

[0017]

[0018] In the formula, X m (t) represents the original input data of the mth mode at time t, represents the generative model of the mth modality, E m represents the feature extractor of the mth modality, α m (t) represents the dynamic weight coefficient of the mth modality changing with time t, F(·) represents the cross-modal information fusion function, Represents the mth generative model G m The parameters of , μ(t) and σ(t) represent the dynamic mean and standard deviation, which are dynamically adjusted over time t and are defined as:

[0019]

[0020] Where W represents the sliding window width, which controls the time dependency.

[0021] Among them, the S3 extracts features from each data modality through an encoder, converts the feature representations of different modalities into a common space, constructs a cross-modal interaction layer, calculates the similarity scores between the modalities based on the results of the cross-modal interaction, adjusts the cross-modal attention weights according to the calculated similarity scores, and optimizes the semantic alignment effect.

[0022] Among them, the S4 preprocesses the data of each modality, directly splices the feature vectors from different modalities, performs late fusion at the decision-making level, processes and classifies the data of each modality, and then integrates the output results of each modality through a voting mechanism and weighted average to create a hybrid fusion framework. According to the changes in modal reliability monitored in real time, the fusion weights are dynamically updated, and the entire fusion system is fully verified using the reserved test set to check whether its performance under different conditions meets expectations.

[0023] Among them, the S5 conducts a comprehensive analysis of the overall task, designs a hierarchical task architecture based on the complexity and diversity of the task, further subdivides the task into smaller, operational subtasks within each level, evaluates the computing resources and storage space requirements for each subtask, and assigns the subtasks to the most suitable agents to execute them based on the professional skills of each agent. It also adjusts resource allocation as needed, establishes a conflict detection mechanism, and implements effective load balancing measures.

[0024] Among them, the S6 sets specific learning goals, selects appropriate reinforcement learning algorithms based on task characteristics and goals, and performs necessary parameter configurations. It integrates privacy protection measures when designing communication protocols, continuously learns the optimal communication strategies through reinforcement learning algorithms, and dynamically adjusts these strategies according to actual operating conditions to enhance the security of overall communication and regularly evaluate the performance of communication protocols.

[0025] Among them, S7 identifies the factors that cause uncertainty in the system, builds a reasoning framework based on Bayesian theorem, extracts features from each modality, and converts them into a form suitable for Bayesian reasoning, designs a multimodal voting mechanism, formulates a set of rules to dynamically adjust the weight of each modality in the final decision, creates an interface where human experts can intervene and review key decision points, and continuously optimizes the performance of the system by collecting feedback and actual results from human experts.

[0026] Among them, the S8 plans the deployment location of edge nodes based on the needs and geographical distribution of application scenarios, implements data preprocessing and preliminary analysis on edge nodes, reduces the amount of data that needs to be transmitted to the cloud, designs an efficient real-time communication architecture so that edge devices can quickly exchange information with central servers or other edge nodes, sets action trigger conditions based on the results of edge computing, builds a real-time feedback collection system, develops online learning algorithms suitable for edge computing environments, establishes performance indicators to monitor the system regularly, and initiates a continuous optimization process based on the collected real-time feedback and performance evaluation results.

[0027] Among them, the S9 collects data sets suitable for different tasks from various sources and performs necessary preprocessing, designs a knowledge distillation mechanism, designs a model architecture based on the selected knowledge distillation strategy and meta-learning algorithm, builds a self-evolving framework that can automatically adjust and optimize its own performance, tests the constructed framework on multiple pre-defined tasks, evaluates its cross-task migration ability and efficiency, and uses a series of indicators to comprehensively evaluate the performance of the entire system.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. This invention uses GAN model adversarial training to fill missing values ​​and remove noise, significantly improving the accuracy of subsequent processing. The dynamic normalization process solves the problem of traditional normalization methods relying on static distribution by tracking data distribution changes in real time. The cross-modal association algorithm maps text, image, and sensor data into a unified space. The continuous monitoring system adjusts parameters through feedback, achieving adaptive preprocessing.

[0030] 2. After extracting features through modality-specific encoders, the cross-modal interaction layer calculates inter-modal similarity through an attention mechanism. Dynamically adjusted attention weights ensure that key modal information is prioritized. The unified space after semantic alignment supports cross-modal retrieval and reasoning. Feature transformation also enhances the model's generalization capability.

[0031] 3. This invention combines early fusion, late fusion, and hybrid strategies to enable flexible adjustments based on task requirements. A dynamic weight update mechanism automatically adjusts the contributions of different modalities. A reserved test set verifies the effectiveness of the fusion strategy under different conditions. Dynamic weight adjustment significantly improves the adaptability of multimodal systems, especially in scenarios with fluctuating data quality or missing modalities, avoiding decision bias caused by the dominance of a single modality.

[0032] 4. The present invention updates the posterior distribution through the prior probability and observation data of the Bayesian framework; the multimodal voting mechanism integrates the outputs of different modalities; dynamic weight adjustment ensures that the high-reliability modality dominates; the human-in-the-loop review interface allows experts to intervene at key decision points and optimize system parameters through feedback loops; the transparency of Bayesian reasoning and the intervention of human feedback significantly enhance the credibility and interpretability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The operation process of the intelligent agent collaboration method based on multimodal data fusion of the present invention is as follows Figure 1 ;

[0034] Figure 2 The operation process of the intelligent agent collaboration method based on multimodal data fusion of the present invention is as follows Figure 2 ;

[0035] Figure 3 The operation process of the intelligent agent collaboration method based on multimodal data fusion of the present invention is as follows Figure 3 ;

[0036] Figure 4 The operation process of the intelligent agent collaboration method based on multimodal data fusion of the present invention is as follows Figure 4 . DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] Example

[0039] See also Figure 1-4 As shown, the present invention provides a technical solution: comprising the following steps:

[0040] S1. Dynamically collect multi-source heterogeneous data, automatically assign agent roles based on reinforcement learning, and optimize task execution efficiency;

[0041] S2. Use the GAN diffusion model to repair low-quality data, combining cross-modal correlation and dynamic normalization to achieve adaptive preprocessing and data quality improvement;

[0042] S3, extract features through modality-specific encoders and use cross-modal attention mechanisms to achieve semantic alignment and eliminate the semantic gap;

[0043] S4, early, late and hybrid fusion strategies, dynamically adjust fusion weights based on modal reliability to balance the contribution of multimodal information;

[0044] S5. Hierarchically plan tasks and decompose them into subtasks. Use resource scheduling and consensus algorithms to coordinate agent collaboration, resolve conflicts, and balance loads.

[0045] S6. Design reinforcement learning-driven communication protocols to optimize communication efficiency and security, and support privacy protection and fault tolerance mechanisms;

[0046] S7. Combine Bayesian reasoning with multimodal voting mechanisms to handle uncertainty, introduce human-in-the-loop review, and improve decision robustness;

[0047] S8, low-latency action execution based on edge computing, continuous optimization of system performance through real-time feedback and online learning;

[0048] S9. Build a self-evolutionary framework, use knowledge distillation and meta-learning to achieve cross-task migration, and improve the long-term adaptability of intelligent agents.

[0049] Among them, in S1, the system first scans and identifies all available data sources, and classifies them according to the characteristics of the data. For each identified data type, the corresponding adapter is dynamically loaded to establish an efficient data flow management system to control the speed and order of data flowing into the system from different sources, define the environment in which the agent operates, create a detailed capability model for each agent, and train the agent using historical data and simulation scenarios. When a new task arrives, the system uses the trained reinforcement learning algorithm to determine the most appropriate agent role allocation plan in real time based on the current environment state, and feeds back the actual results to the reinforcement learning module during the agent's task execution.

[0050] Among them, S2 performs a preliminary quality assessment on the collected multi-source heterogeneous data, identifies noise, missing values, and outliers in the data, selects an appropriate generative adversarial network (GAN) model for data repair based on the data type, applies the selected GAN model to predict and fill in the missing parts in the data, uses the GAN model to remove unnecessary interference signals for noisy data sets, detects and corrects outliers in the data, extracts features from different types of input data, and attempts to find the correlation between them, uses appropriate algorithms for cross-modal information fusion, implements a dynamic standardization process, establishes a continuous monitoring and feedback system to track the data processing effect, and adjusts the parameter settings in the previous steps accordingly. The implementation formula is:

[0051]

[0052] In the formula, X m (t) represents the original input data of the mth mode at time t, represents the generative model of the mth modality, E m represents the feature extractor of the mth modality, α m (t) represents the dynamic weight coefficient of the mth modality changing with time t, F(·) represents the cross-modal information fusion function, Represents the mth generation model G m The parameters of , μ(t) and σ(t) represent the dynamic mean and standard deviation, which are dynamically adjusted over time t and are defined as:

[0053]

[0054] Where W represents the sliding window width, which controls the time dependency.

[0055] Among them, the S3 extracts features from each data modality through an encoder, converts the feature representations of different modalities into a common space, constructs a cross-modal interaction layer, calculates the similarity scores between the modalities based on the results of the cross-modal interaction, adjusts the cross-modal attention weights according to the calculated similarity scores, and optimizes the semantic alignment effect.

[0056] Among them, the S4 preprocesses the data of each modality, directly splices the feature vectors from different modalities, performs late fusion at the decision-making level, processes and classifies the data of each modality, and then integrates the output results of each modality through a voting mechanism and weighted average to create a hybrid fusion framework. According to the changes in modal reliability monitored in real time, the fusion weights are dynamically updated, and the entire fusion system is fully verified using the reserved test set to check whether its performance under different conditions meets expectations.

[0057] Among them, the S5 conducts a comprehensive analysis of the overall task, designs a hierarchical task architecture based on the complexity and diversity of the task, further subdivides the task into smaller, operational subtasks within each level, evaluates the computing resources and storage space requirements for each subtask, and assigns the subtasks to the most suitable agents to execute them based on the professional skills of each agent. It also adjusts resource allocation as needed, establishes a conflict detection mechanism, and implements effective load balancing measures.

[0058] Among them, the S6 sets specific learning goals, selects appropriate reinforcement learning algorithms based on task characteristics and goals, and performs necessary parameter configurations. It integrates privacy protection measures when designing communication protocols, continuously learns the optimal communication strategies through reinforcement learning algorithms, and dynamically adjusts these strategies according to actual operating conditions to enhance the security of overall communication and regularly evaluate the performance of communication protocols.

[0059] Among them, S7 identifies the factors that cause uncertainty in the system, builds a reasoning framework based on Bayesian theorem, extracts features from each modality, and converts them into a form suitable for Bayesian reasoning, designs a multimodal voting mechanism, formulates a set of rules to dynamically adjust the weight of each modality in the final decision, creates an interface where human experts can intervene and review key decision points, and continuously optimizes the performance of the system by collecting feedback and actual results from human experts.

[0060] Among them, the S8 plans the deployment location of edge nodes based on the needs and geographical distribution of application scenarios, implements data preprocessing and preliminary analysis on edge nodes, reduces the amount of data that needs to be transmitted to the cloud, designs an efficient real-time communication architecture so that edge devices can quickly exchange information with central servers or other edge nodes, sets action trigger conditions based on the results of edge computing, builds a real-time feedback collection system, develops online learning algorithms suitable for edge computing environments, establishes performance indicators to monitor the system regularly, and initiates a continuous optimization process based on the collected real-time feedback and performance evaluation results.

[0061] Among them, the S9 collects data sets suitable for different tasks from various sources and performs necessary preprocessing, designs a knowledge distillation mechanism, designs a model architecture based on the selected knowledge distillation strategy and meta-learning algorithm, builds a self-evolving framework that can automatically adjust and optimize its own performance, tests the constructed framework on multiple pre-defined tasks, evaluates its cross-task migration ability and efficiency, and uses a series of indicators to comprehensively evaluate the performance of the entire system.

[0062] Working principle: First, scan and identify all available data sources, dynamically load adapters according to data types, establish a data flow management system to control the rate and priority of data inflow, and through reinforcement learning, the intelligent agent assigns roles in real time according to the current environment status; the system evaluates the quality of multi-source data, identifies noise, missing values ​​or outliers, selects dedicated GAN models for different data types, fills in missing parts through adversarial training, removes noise, and detects outliers; for different modalities, dedicated encoders extract features and map the features to a shared semantic space, dynamically adjust attention weights, optimize semantic alignment effects, and eliminate semantic gaps between modalities; adopts early fusion, late fusion and hybrid strategies, flexibly switches according to task requirements, and monitors modal availability in real time Reliability, dynamically update fusion weights; decompose complex tasks into hierarchical subtasks, and dynamically allocate them based on the skills of the agents. The resource scheduling module adjusts resource allocation according to computing needs, and the conflict detection mechanism prevents multiple agents from competing for the same resources; design communication strategies through reinforcement learning, and dynamically adjust the frequency and path of data transmission; build a Bayesian framework, convert multimodal features into probability distributions, and dynamically update the posterior probability; deploy data preprocessing at edge nodes to reduce the amount of data transmitted from the cloud; deploy data preprocessing at edge nodes to reduce the amount of data transmitted from the cloud; migrate the knowledge of complex models to lightweight models through knowledge distillation to reduce computing costs, meta-learning optimizes initialization parameters through cross-task experience, and the automatic adjustment of the self-evolution framework reduces manual intervention.

[0063] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

[0064] The present invention and its embodiments are described above. This description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs structures and embodiments similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. An agent collaboration method based on multimodal data fusion, characterized in that: The following steps are involved: S1. Dynamically collect multi-source heterogeneous data, automatically assign agent roles based on reinforcement learning, and optimize task execution efficiency; S2. Use the GAN diffusion model to repair low-quality data, combining cross-modal correlation and dynamic normalization to achieve adaptive preprocessing and data quality improvement; S3, extract features through modality-specific encoders and use cross-modal attention mechanisms to achieve semantic alignment and eliminate the semantic gap; S4, early, late and hybrid fusion strategies, dynamically adjust fusion weights based on modal reliability to balance the contribution of multimodal information; S5. Hierarchically plan tasks and decompose them into subtasks. Use resource scheduling and consensus algorithms to coordinate agent collaboration, resolve conflicts, and balance loads. S6. Design reinforcement learning-driven communication protocols to optimize communication efficiency and security, and support privacy protection and fault tolerance mechanisms; S7. Combine Bayesian reasoning with multimodal voting mechanisms to handle uncertainty, introduce human-in-the-loop review, and improve decision robustness; S8, low-latency action execution based on edge computing, continuous optimization of system performance through real-time feedback and online learning; S9. Build a self-evolutionary framework, use knowledge distillation and meta-learning to achieve cross-task migration, and improve the long-term adaptability of intelligent agents.

2. The agent collaboration method based on multimodal data fusion according to claim 1, characterized in that: In S1, the system first scans and identifies all available data sources and classifies them according to the characteristics of the data. For each identified data type, the corresponding adapter is dynamically loaded to establish an efficient data flow management system to control the speed and order of data flowing into the system from different sources, define the environment in which the agent operates, create a detailed capability model for each agent, and train the agent using historical data and simulation scenarios. When a new task arrives, the system uses the trained reinforcement learning algorithm to determine the most appropriate agent role allocation plan in real time based on the current environment state, and feeds back the actual results to the reinforcement learning module during the agent's task execution.

3. The agent collaboration method based on multimodal data fusion according to claim 1, characterized in that: S2 performs a preliminary quality assessment on the collected multi-source heterogeneous data, identifies noise, missing values, and outliers in the data, selects an appropriate generative adversarial network (GAN) model for data repair based on the data type, applies the selected GAN model to predict and fill in the missing parts in the data, uses the GAN model to remove unnecessary interference signals for noisy data sets, detects and corrects outliers in the data, extracts features from different types of input data, and attempts to find correlations between them, uses appropriate algorithms for cross-modal information fusion, implements a dynamic standardization process, establishes a continuous monitoring and feedback system to track the data processing effect, and adjusts the parameter settings in the previous steps accordingly. The implementation formula is: In the formula, X m (t) represents the original input data of the mth mode at time t, represents the generative model of the mth modality, E m represents the feature extractor of the mth modality, α m (t) represents the dynamic weight coefficient of the mth modality changing with time t, F(·) represents the cross-modal information fusion function, Represents the mth generative model G m The parameters of , μ(t) and σ(t) represent the dynamic mean and standard deviation, which are dynamically adjusted over time t and are defined as: Where W represents the sliding window width, which controls the time dependency.

4. The agent collaboration method based on multimodal data fusion according to claim 1, characterized in that: The S3 extracts features from each data modality through an encoder, converts the feature representations of different modalities into a common space, constructs a cross-modal interaction layer, calculates the similarity scores between the modalities based on the results of the cross-modal interaction, adjusts the cross-modal attention weights according to the calculated similarity scores, and optimizes the semantic alignment effect.

5. The agent collaboration method based on multimodal data fusion according to claim 1, characterized in that: The S4 preprocesses the data of each modality, directly splices the feature vectors from different modalities, performs late fusion at the decision-making level, processes and classifies the data of each modality, and then integrates the output results of each modality through a voting mechanism and weighted average to create a hybrid fusion framework. According to the changes in modal reliability monitored in real time, the fusion weights are dynamically updated, and the entire fusion system is fully verified using a reserved test set to check whether its performance under different conditions meets expectations.

6. The agent collaboration method based on multimodal data fusion according to claim 1, characterized in that: The S5 conducts a comprehensive analysis of the overall task and designs a hierarchical task architecture based on the complexity and diversity of the task. Within each level, the task is further subdivided into smaller, actionable subtasks. The computing resources and storage space requirements required for each subtask are evaluated. Based on the professional skills of each intelligent agent, the subtasks are assigned to the intelligent agent that is most suitable to execute them. Resource allocation is adjusted as needed, a conflict detection mechanism is established, and effective load balancing measures are implemented.

7. The agent collaboration method based on multimodal data fusion according to claim 1, characterized in that: Said S6 sets specific learning goals, selects appropriate reinforcement learning algorithms according to task characteristics and goals, and performs necessary parameter configurations. Privacy protection measures are integrated when designing communication protocols. The optimal communication strategies are continuously learned through reinforcement learning algorithms, and these strategies are dynamically adjusted according to actual operation conditions to enhance the security of overall communication and regularly evaluate the performance of communication protocols.

8. The agent collaboration method based on multimodal data fusion according to claim 1, characterized in that: Said S7 identifies the factors causing uncertainty in the system, constructs a reasoning framework based on Bayesian theorem, extracts features from each modality and converts them into a form suitable for Bayesian reasoning, designs a multimodal voting mechanism, and formulates a set of rules to dynamically adjust the weight of each modality in the final decision.

9. The agent collaboration method based on multimodal data fusion according to claim 1, characterized in that: The S8 plans the deployment location of edge nodes based on the needs and geographical distribution of application scenarios, implements data preprocessing and preliminary analysis on edge nodes, reduces the amount of data that needs to be transmitted to the cloud, designs an efficient real-time communication architecture so that edge devices can quickly exchange information with central servers or other edge nodes, sets action trigger conditions based on the results of edge computing, builds a real-time feedback collection system, develops online learning algorithms suitable for edge computing environments, establishes performance indicators to monitor the system regularly, and initiates a continuous optimization process based on the collected real-time feedback and performance evaluation results.

10. The agent collaboration method based on multimodal data fusion according to claim 1, characterized in that: The S9 collects data sets suitable for different tasks from various sources and performs necessary preprocessing, designs a knowledge distillation mechanism, designs a model architecture based on the selected knowledge distillation strategy and meta-learning algorithm, builds a self-evolving framework that can automatically adjust and optimize its own performance, tests the constructed framework on multiple predefined tasks, evaluates its cross-task migration capability and efficiency, and uses a series of indicators to comprehensively evaluate the performance of the entire system.

Images