An adaptive hybrid expert model training framework for multi-agent systems

Abstract

The present invention provides an adaptive hybrid expert model training framework for a multi-agent system, comprising a training component, a large language model adaptive control component, and front-end and back-end system components. The training component is used to preprocess input data and dynamically select expert models, which are selected through a preset gating mechanism. The large language model adaptive control component is used to optimize the training component. The front-end and back-end system components serve as interfaces for user interaction with the system, providing a front-end user interaction interface and a back-end task management system module. This adaptive hybrid expert model training framework for a multi-agent system exhibits significant advantages in terms of expert model diversity, flexible gating mechanisms, user customization capabilities, and real-time training feedback.

Description

Technical Field

[0001] The present invention relates to a hybrid expert model, and in particular to an adaptive hybrid expert model training framework for a multi-agent system. Background Art

[0002] In the current field of multi-agent systems and artificial intelligence (AI), deep learning models, particularly large language models (LLMs), have achieved remarkable progress in fields such as natural language processing (NLP), computer vision, and reinforcement learning. These models rely on a large parameter size and extensive computing resources to capture complex patterns and structures. However, as the model parameter size increases, the cost of training and inference also increases exponentially. This poses new challenges for how to effectively train and deploy these models.

[0003] In existing technologies, the Mixture of Experts (MoE) model has gradually become an effective solution. It aims to reduce computational burden and memory usage by selectively activating some expert networks within the model, thereby improving model training efficiency and inference speed. The MoE model introduces a gating mechanism to dynamically select activated expert models based on the characteristics of the input data, avoiding computational redundancy across the entire model. Typical MoE models, such as Google's SwitchTransformer and Microsoft's DeepSpeed-MoE, have significantly improved the performance of large-scale models through this mechanism.

[0004] However, the existing MoE model architecture also faces some limitations:

[0005] (1) Single choice of expert model: Most existing MoE models are limited to using Transformer experts and lack a variety of expert model choices, making them inflexible when dealing with different task types.

[0006] (2) Fixed gating mechanism: Many MoE models use fixed Top-k Gating or Soft Gating. Although these mechanisms improve computational efficiency to a certain extent, they are insufficient in terms of model generalization ability and ability to cope with complex tasks.

[0007] (3) Limited user customization capabilities: In existing MoE models, user customization options are relatively limited and usually rely on preset optimization schemes, making it difficult to flexibly adjust the model architecture and training strategies according to different tasks or needs.

[0008] With the development of AI technology, more and more application scenarios require not only strong model performance but also efficient use of computing resources and adaptability to different types of tasks. For example, in large-scale multimodal systems, structured data, unstructured text, and image processing may be involved simultaneously, which places higher demands on model flexibility and customization. Traditional MoE models have shown certain limitations in meeting these diverse requirements. Summary of the Invention

[0009] The present invention provides an adaptive hybrid expert model training framework for multi-agent systems. By improving the existing MoE model architecture, it provides a more flexible and efficient solution that can better adapt to diverse task requirements and provide efficient training and inference support in environments with limited computing resources. It is used to solve the problem of providing users with a highly customizable MoE model training and inference platform by combining different types of expert models (such as FNN, Transformer, hybrid expert model) and multiple gating mechanisms (such as Soft Gating, Top-k Gating, Noisy Top-k Gating), thereby improving the performance and generalization ability of the model. The technical solution is as follows:

[0010] An adaptive hybrid expert model training framework for a multi-agent system includes a training part, a large language model adaptive regulation part, and front-end and back-end system parts. The training part is used to preprocess input data and dynamically select expert models, and the expert models are selected through a preset gating mechanism. The large language model adaptive regulation part is used to optimize the training part. The front-end and back-end system parts serve as interfaces for user-system interaction, providing a front-end user interaction interface and a back-end task management system module.

[0011] The training part includes a data preprocessing module, a gating mechanism module, an expert model module and a parameter optimization module. The data preprocessing module is used to preprocess the input data. The gating mechanism module dynamically selects the expert model for the preprocessed data according to the preset gating mechanism. The expert model module activates the corresponding expert model for calculation through the expert model selected by the gating mechanism module. Finally, the parameter optimization module updates the parameters of the expert model through the back propagation algorithm, optimizes the loss function and improves the performance of the expert model.

[0012] The large language model adaptive control part includes a routing control module for dynamically adjusting the expert model activated by the MoE model; and a model monitoring and adjustment module for adaptively adjusting the parameters of the MoE model architecture and the expert model based on the monitoring results.

[0013] The large language model adaptive control part also includes an expert scheduling module for scheduling expert models in the activated expert models.

[0014] Through the Django backend combined with statistical analysis tools, the MoE model system collects various indicators during the training process. The monitoring data is analyzed through the control algorithm, and the system automatically adjusts the activation ratio, gating strategy and parameter optimization method of the expert model. The model monitoring and adjustment module is used to monitor the performance of the expert model during the training process, including loss, accuracy, and computing resource usage, and adaptively adjust the model architecture and parameters based on the monitoring results.

[0015] The front-end and back-end parts include a front-end user interface, a back-end task management system module, and a database and storage management module. The front-end user interface provides users with a convenient interface for model configuration, training progress viewing, and model performance analysis; the back-end task management system module supports training tasks submitted by users, and is responsible for coordinating the interaction between the front-end and the training system, and is also responsible for interacting with the database to save training records, model configurations, and results; the database and storage management module is used to save training data, model parameters, training logs, and results.

[0016] The expert model can use a feedforward neural network, a Transformer model, a hybrid expert model, a convolutional neural network, a long short-term memory network, or an autoregressive model.

[0017] The gating mechanism module adopts soft gating, top-k gating, noisy top-k gating, hard gating, evolutionary gating, and gating mechanism based on reinforcement learning.

[0018] The large language model adaptive control part can be replaced by rule-driven adaptive control or adaptive control based on statistical analysis.

[0019] The training part can be replaced by a meta-learning-based training strategy, a self-supervised learning training method, or mixed precision training.

[0020] The adaptive hybrid expert model training framework for multi-agent systems, based on an innovative solution to the Mixture of Experts (MoE) model architecture, provides a flexible, efficient, and customizable large-scale model training and inference platform. Compared with existing technologies, the present invention demonstrates significant advantages in terms of expert model diversity, flexible gating mechanisms, user customization capabilities, and real-time training feedback. Analysis of the present invention's technical solutions reveals the following technical effects and advantages, and further explains their causes through reasonable reasoning.

[0021] (1) Expert model diversity improves task adaptability

[0022] Limitations of Existing Technologies: Current MoE model architectures are typically limited to a single type of expert model, such as the Transformer, which has limited performance when handling complex or diverse tasks. These models are less effective when handling non-linguistic tasks, such as structured data or time series analysis, resulting in insufficient adaptability. The "expert network" or "expert model" in MoE refers to the "E" in the MoE, meaning that multiple expert models form a hybrid expert model.

[0023] Improvements and Effects of the Invention: By integrating multiple types of expert models, including feedforward neural networks (FNNs), transformers, and hybrid expert models, the present invention enables the MoE model system to flexibly select different expert models based on task characteristics. FNNs excel at processing linear tasks, while transformers excel at complex sequence tasks. Hybrid expert models can comprehensively leverage the advantages of different models to handle complex data. The system serves as a training, deployment, and deployment system for MoE models.

[0024] This diverse selection of expert models makes the system applicable not only to natural language processing (NLP) tasks, but also to other application scenarios such as financial data analysis, image processing, and time series data forecasting. By providing a wider range of expert model options, the system can better handle different types of tasks, significantly improving task adaptability and model expressiveness.

[0025] (2) Flexible gating mechanism improves computational efficiency and model generalization capability

[0026] Limitations of existing technologies: Existing MoE model architectures mostly use fixed gating mechanisms, such as Top-k Gating or Soft Gating. While these fixed gating strategies offer some optimization in terms of computational resources, they lack the ability to generalize models and handle diverse tasks.

[0027] Improvements and Effects of the Present Invention: This present invention designs flexible gating mechanisms, including Soft Gating, Top-k Gating, and Noisy Top-k Gating. In particular, the Noisy Top-k Gating mechanism introduces randomness, making the expert model activated each time not completely fixed. This increases the diversity of expert model selection, thereby improving the model's generalization ability and avoiding over-reliance on a specific expert model.

[0028] This flexible gating mechanism enables the system to improve the model's generalization capabilities without sacrificing computational efficiency. This is particularly true in scenarios with uneven data distribution or high task complexity. The introduction of random noise enables the MoE model to avoid overfitting to certain expert models, enhancing its stability and robustness across diverse task scenarios. Furthermore, through the combination of Top-k Gating and Noisy Top-k Gating, the system effectively utilizes computing resources while reducing the computational burden, achieving a balance between computational efficiency and generalization capabilities.

[0029] (3) User customization capabilities enhance system flexibility and controllability

[0030] Limitations of existing technologies: Traditional MoE model architectures typically offer limited customization options, making it difficult for users to flexibly adjust expert models and gating strategies based on task requirements. This results in a lack of fine-grained control over the model when handling different tasks, preventing the system from fully realizing its potential.

[0031] The improvements and effects of this invention: By allowing users to select expert models and gating mechanisms, users can flexibly configure the model architecture and training process based on specific tasks. Users can not only select different expert models, but also freely choose the appropriate gating mechanism, such as Soft Gating, Top-k Gating, or Noisy Top-k Gating.

[0032] This customization capability enables the system to better adapt to complex or specialized task requirements. For example, users can choose the FNN model for efficient linear tasks, the Transformer for language tasks, and a hybrid expert model for multimodal data. Furthermore, users can dynamically adjust gating strategies based on computing resource constraints and task generalization requirements, maximizing system flexibility and controllability. This highly customized design significantly improves the system's applicability and user experience across diverse application scenarios.

[0033] (4) Real-time analysis and training feedback of large language models improve training efficiency and effectiveness

[0034] Limitations of existing technologies: In the traditional MoE model training process, model training feedback is usually processed offline or in batches, making it impossible to analyze and adjust the model performance in real time during training. This approach may lead to low training efficiency or overfitting problems.

[0035] Improvements and Effects of the Presentation: By integrating a large language model (LLM), the system can analyze model performance in real time during training and provide feedback and recommendations based on the analysis results. The system can dynamically adjust training parameters, such as the learning rate, expert model selection strategy, and the number of activated expert models, based on the current loss curve, expert model selection, and resource usage, thereby optimizing the training process.

[0036] By analyzing large language models in real time, the system automatically identifies model deficiencies during training and makes timely adjustments during intermediate stages. This real-time feedback mechanism enables the system to correct overfitting, bias, or resource misallocation issues at an early stage, avoiding wasted training time and computing resources. Furthermore, by dynamically adjusting model parameters, the system can identify the optimal expert model selection strategy and gating mechanism at different stages, thereby improving training efficiency and ensuring that the model achieves optimal performance with minimal resource consumption.

[0037] (5) Resource utilization efficiency has been greatly improved

[0038] Limitations of existing technologies: The training and inference of large-scale models usually require huge computing resources and memory usage, especially in resource-constrained environments (such as embedded devices or edge computing scenarios), where traditional MoE models are difficult to deploy efficiently.

[0039] Improvements and Effects of the Presentation: This present invention reduces the number of expert models activated during each training or inference process through Top-k Gating and Noisy Top-k Gating, thereby reducing the system's computational and memory consumption. Furthermore, by introducing an adaptive control module, the system can dynamically adjust the allocation of computing resources based on current task requirements, avoiding unnecessary resource waste.

[0040] By reducing the number of expert model activations, this system significantly reduces computational requirements during training and inference. This selective activation strategy significantly reduces memory usage and computational burden, especially when working with large-scale models. The adaptive control module further ensures efficient deployment in environments with limited computing resources, while also avoiding excessive resource consumption through rational resource allocation. This design significantly improves the system's resource utilization, making it more suitable for deployment in diverse computing environments.

[0041] In summary, the present invention demonstrates significant advantages over existing MoE architectures through numerous technical improvements. By introducing diverse expert models, flexible gating mechanisms, user customization capabilities, and real-time training feedback for large language models, the system can provide improved task adaptability, higher training efficiency, and stronger generalization capabilities based on efficient utilization of computing resources. These technical effects, through rational design and innovative implementation methods, effectively address key issues in existing technologies and enhance the practical application performance of large-scale models in training and reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the adaptive hybrid expert model training framework for the multi-agent system. DETAILED DESCRIPTION

[0043] like Figure 1 As shown, the adaptive hybrid expert model training framework for the multi-agent system consists of a training component, a large language model adaptive control component, and front-end and back-end systems. The training component is used to preprocess input data and dynamically select expert models, which are selected through a preset gating mechanism; the large language model adaptive control component is used to optimize the training component; and the front-end and back-end systems serve as the interface for user interaction with the MoE model, providing a front-end user interface and a back-end task management system module. Each component plays a key role in the overall system workflow, from preprocessing training data, dynamically adjusting the expert model, to presenting the user interface, ensuring efficient model training and inference.

[0044] Training: This core component of the present invention is responsible for preprocessing input data, selecting expert models, and optimizing model parameters. It includes a data preprocessing module, a gating mechanism module, an expert model module, and a parameter optimization module. The data preprocessing module preprocesses input data. The gating mechanism module dynamically selects expert models for preprocessed data based on a preset gating mechanism. The expert model module activates the corresponding expert model for calculations based on the expert model selected by the gating mechanism module. Finally, the parameter optimization module updates the expert model parameters using a backpropagation algorithm, optimizing the loss function and improving expert model performance.

[0045] The modules mentioned above are specifically described as follows:

[0046] 1. Data preprocessing module, which preprocesses the input data, including text cleaning, data enhancement, feature extraction, etc. The preprocessed data will be passed to the gating mechanism module as preprocessed data.

[0047] Preprocessing can utilize Python's multi-threaded processing and data processing libraries such as Pandas, combined with natural language processing techniques (such as Tokenization, word embedding, etc.) to preprocess input data.

[0048] The gating mechanism module calculates the contribution of each expert model through a gating network, typically implemented as a small neural network. In Soft Gating, all expert models participate in the calculation, and the final output is a weighted sum. In Top-k Gating, only the k highest-scoring expert models are selected. Noisy Top-k Gating introduces noise to randomize the selection of expert models, increasing model diversity and generalization.

[0049] The gating mechanism module dynamically selects the expert model for preprocessed data according to a preset gating mechanism (Soft Gating, Top-k Gating or NoisyTop-k Gating). Different gating mechanisms correspond to different computational burdens and generalization capabilities.

[0050] 3 expert model modules, each expert model exists independently during design. FNN is good at processing linear tasks, Transformer is suitable for complex sequence tasks, and the hybrid expert model can comprehensively utilize the advantages of different networks. The training part uses PyTorch or DeepSpeed ​​framework for parallel acceleration and optimization.

[0051] The expert model module activates the corresponding expert model (FNN, Transformer, or hybrid expert model) for calculation based on the results selected by the gating mechanism module. Each expert model is optimized for a specific task type or data feature to improve computational efficiency and model performance.

[0052] 4. Parameter optimization module, based on the automatic differentiation method, combined with optimization algorithms (such as Adam or SGD), realizes automatic optimization of the parameters of the expert model. The update of the parameters is limited to the activated expert model, thereby reducing unnecessary computing resource consumption.

[0053] In the training part, the parameter optimization module updates the parameters of the expert model through the back-propagation algorithm, optimizes the loss function and improves the performance of the expert model.

[0054] The second large language model adaptive control section: The large language model adaptive control section is the intelligent control part of the present invention, responsible for adaptively optimizing the MoE model architecture and expert model through training and inference to ensure that the system can dynamically adjust the model to achieve optimal performance in different task scenarios. It includes a routing control module and a model monitoring and adjustment module, which are respectively used to dynamically adjust the expert model activated by the MoE model and adaptively adjust the parameters of the MoE model architecture and expert model based on the monitoring results. In addition, it also includes an expert scheduling module for rationally scheduling the expert models in the activated expert models.

[0055] 1. Routing control module, which uses reinforcement learning or a rule-based approach to dynamically adjust the expert model selection strategy at each inference or training step. In practice, the routing network selects the expert model that provides the best performance based on current input features and historical performance.

[0056] The routing control module dynamically adjusts the expert models activated in the MoE model by monitoring the characteristics of the input data in real time. By adaptively adjusting different expert models, the MoE model system can select the optimal expert model combination based on different input conditions.

[0057] The model monitoring and adjustment module uses the Django backend combined with statistical analysis tools to collect various metrics during the training process. After analyzing this monitoring data using a control algorithm (such as Bayesian optimization or genetic algorithm), the system can automatically adjust the expert model's activation ratio, gating strategy, and parameter optimization method.

[0058] The model monitoring and adjustment module monitors the performance of the expert model during training (e.g., loss, accuracy, and computing resource usage) and adaptively adjusts the model architecture and parameters based on the monitoring results. The system dynamically changes the gating strategy and expert model selection based on task requirements.

[0059] 3. Expert Scheduling Module: Based on a scheduling algorithm (such as round-robin or priority-based scheduling), the system schedules activated expert models appropriately. DeepSpeed ​​or other distributed training libraries are used to optimize computing task scheduling across multiple GPUs and nodes.

[0060] This module is responsible for intelligent scheduling in a multi-expert model environment, ensuring that the system can efficiently allocate computing resources under multiple parallel tasks, avoiding excessive use of expert models or waste of computing resources.

[0061] The front-end and back-end components are the interfaces between users and the system. They provide an intuitive and easy-to-use user interface and support the scheduling, monitoring, and visualization of training results. These components include the front-end user interface, the back-end task management system, and the database and storage management module.

[0062] 1. Front-end user interface, developed using Javascript (such as React or Vue), combined with data visualization libraries such as D3.js or Plotly.js to display key data during the training process, such as loss curves, expert model selection ratios, etc.

[0063] The front-end user interface provides users with a convenient interface for model configuration, training progress viewing, and model performance analysis. Users can use the graphical interface to select training tasks, configure expert models and gating mechanisms, and view the training progress and performance indicators of the MoE model.

[0064] The backend task management system module, built on the Django framework, receives model configuration requests from users and manages the scheduling of model training tasks through task queues (such as Celery). Django interacts with the PyTorch / DeepSpeed ​​libraries to control the training process.

[0065] The backend task management system module supports training tasks submitted by users and is responsible for coordinating the interaction between the front end and the training system; it is also responsible for interacting with the database to save training records, model configurations and results.

[0066] The database and storage management module can connect to PostgreSQL or other relational databases for data storage and integrate with Django ORM for data management and query. The intermediate results and final model weights of the model are stored and managed using a distributed file system.

[0067] The database and storage management module can save training data, model parameters, training logs and results, ensuring that users can access training history and results at any time.

[0068] Furthermore, the present invention may use other alternatives for the expert model, gating mechanism, replacement of the adaptive control part, training part, and front-end and back-end parts.

[0069] 1. Alternatives to Expert Models

[0070] Original Solution: This paper uses three types of expert models: feedforward neural networks (FNNs), transformers, and hybrid expert models, optimizing for linear tasks, complex contextual tasks, and multimodal tasks, respectively. Alternative solutions include convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and autoregressive models.

[0071] Convolutional neural networks (CNNs) are highly effective as expert models for image processing and other tasks requiring spatial feature capture. Therefore, CNNs can be used as an expert model to replace FNNs or Transformers, making them particularly suitable for computer vision and image classification tasks. By introducing CNNs, the present invention can be further extended to multimodal data processing, including the integration of diverse data sources such as images, text, and audio.

[0072] Long Short-Term Memory (LSTM) as an Expert Model: For time series analysis or tasks requiring processing long-term dependencies, LSTM or its variants (such as GRU) can replace FNN or Transformer models as expert models specifically designed for processing time series data. This alternative can improve model performance in areas such as financial data forecasting and sensor data analysis.

[0073] Autoregressive models as expert models: In some language tasks, autoregressive models (such as GPT variants) can be introduced to replace the standard Transformer model. Autoregressive models are suitable for tasks that require the gradual generation of prediction values, and are particularly advantageous in text generation and machine translation.

[0074] 2. Alternatives to gating mechanisms

[0075] Original Solution: This paper proposes three gating mechanisms, including Soft Gating, Top-k Gating, and Noisy Top-k Gating, to optimize the selection of expert models and the utilization of computing resources. Alternative solutions include hard gating, reinforcement learning-based gating, and evolutionary gating.

[0076] Hard Gating: Unlike soft gating, hard gating activates only the highest-scoring expert model and ignores other expert models. While hard gating may reduce generalization in some tasks, it significantly reduces the computational burden and is particularly suitable for scenarios where inference speed is critical. This alternative can be used in extremely resource-constrained or low-power devices.

[0077] Reinforcement Learning-Based Gating Mechanism: In an alternative approach, the gating mechanism can be dynamically adjusted using reinforcement learning strategies. The system can adaptively select the most appropriate expert model by learning task requirements in real time. This approach can further optimize the expert model selection strategy and iteratively adjust it based on model performance and feedback, improving the accuracy and flexibility of expert model selection.

[0078] Evolutionary Gating: By using evolutionary algorithms (such as genetic algorithms) to select expert models, the system can gradually optimize model performance by "evolutionarily" selecting expert model combinations. Each selected expert model combination can be adjusted based on previous performance, ensuring continuous model improvement.

[0079] 3. Alternatives to the Adaptive Control Section

[0080] Original Solution: This invention uses a large language model to analyze and control the training process in real time, providing an automatic adjustment mechanism based on model feedback. Alternative solutions can use rule-driven adaptive control or statistical analysis-based adaptive control as the adaptive control component.

[0081] Rule-driven adaptive control: An alternative to large language model analysis is to implement adaptive control through a predefined rule system. For example, users can define different expert model selection strategies and gating mechanisms based on different stages of the task. The system dynamically adjusts the training process based on preset conditions (such as the convergence rate of the loss function and resource usage), without relying on real-time analysis of the large language model.

[0082] Adaptive control based on statistical analysis: The system can use statistical analysis tools to evaluate model performance in real time and adjust model training parameters based on simple statistical indicators (such as mean error and variance). This approach can reduce implementation complexity while still enabling dynamic control of the training process.

[0083] 4. Alternatives to the training part

[0084] Original solution: Training involves updating model parameters using traditional backpropagation algorithms (such as Adam and SGD), combined with distributed computing frameworks (such as DeepSpeed) to accelerate training. Alternative solutions include using meta-learning-based training strategies, self-supervised learning, and mixed-precision training.

[0085] Meta-learning-based training strategies: The system can use meta-learning algorithms to quickly adjust model parameters as tasks change, thereby accelerating the model's adaptability. Meta-learning can help the model quickly learn the optimal strategy with a small number of samples. This alternative approach is particularly suitable for online learning or incremental learning tasks.

[0086] Self-supervised learning: This allows the system to be trained without labeled data, leveraging the input data itself to generate learning objectives. This approach can effectively improve model performance even when labeled data is insufficient, without relying on large, manually labeled datasets.

[0087] Mixed-precision training: Using mixed-precision training technology can further accelerate the training process of large-scale models. By converting some computational tasks to half-precision floating-point numbers (FP16), the system can reduce the computational burden and improve training efficiency while maintaining model accuracy.

[0088] 5. Alternatives for front-end and back-end

[0089] Original Solution: This invention uses a JavaScript-based (e.g., React, Vue) front-end user interface, a Django framework back-end for interaction with a task management system, and a distributed file system for data and model storage. Alternative solutions may use other front-end, back-end, and storage alternatives.

[0090] Front-end alternatives: The front-end user interface can be implemented using other frameworks, such as Angular or Svelte. Alternatively, the user interface can be provided through a command-line interface (CLI) or a simplified mobile app. In some use cases, a simple CLI is more suitable for specific developers or operators than a graphical interface.

[0091] Backend Alternatives: The backend can use alternative frameworks to Django, such as Flask, FastAPI, or Node.js. Different backend frameworks can adapt to different scalability and performance requirements. Furthermore, the task scheduling system can replace the existing Celery with another message queue implementation, such as Apache Kafka or RabbitMQ, to support more complex parallel processing and task scheduling.

[0092] Storage alternatives: Alternative file systems or cloud storage solutions can be used for storage. For example, alternatives to HDFS or S3 could include Google Cloud Storage, Azure Blob Storage, and others. Furthermore, for local deployments, the system can use a local database (such as SQLite) to store training logs and model data, further simplifying the system's complexity.

[0093] The present invention has the following characteristics:

[0094] 1. A diverse expert model selection mechanism. This invention integrates FNN (feedforward neural network), Transformer, and hybrid expert models into the MoE model architecture, breaking the limitation of a single expert model in existing technologies and allowing the system to select the most appropriate expert model based on task requirements. By providing a wider range of expert model options, this invention significantly improves the system's task adaptability and ability to process multimodal data, expanding the application of the MoE model architecture in different fields.

[0095] 2. Flexible gating mechanism design. This invention uses three gating mechanisms: Soft Gating, Top-k Gating, and Noisy Top-k Gating. This allows users to flexibly select one based on task requirements. Noisy Top-k Gating introduces randomness, improving the model's generalization capabilities. Compared to existing fixed gating mechanisms, this flexible gating design not only improves the model's computational efficiency but also enhances its generalization capabilities by introducing randomness, reducing the risk of overfitting and providing greater adaptability.

[0096] 3. Real-time Adaptive Control Module: This invention integrates a large language model (LLM) or a rule-based adaptive control module that monitors and analyzes the training process in real time and automatically adjusts the selection of expert models, gating strategies, and training parameters based on feedback. Through real-time monitoring and automatic adjustment, this invention significantly improves the efficiency and effectiveness of the training process, avoiding the delays and resource waste caused by manual adjustments in traditional systems, enabling the model to quickly adapt to task changes and ensure optimal performance.

[0097] 4. User Customization and Control. This invention offers a high degree of user customization, allowing users to independently select expert models and gating mechanisms, adjust model structure and training strategies, and optimize model performance for different task scenarios. Compared to existing technologies, this user customization significantly enhances system flexibility, allowing users to dynamically adjust the model based on task requirements, making it suitable for a wide range of application scenarios, from small-scale to large-scale, complex tasks.

[0098] 5. Efficient computing resource management and utilization. This invention reduces the number of expert models activated during each training and inference phase through Top-k Gating and Noisy Top-k Gating, while also introducing a dynamic resource management mechanism to ensure efficient utilization of computing resources. This invention can significantly reduce computing requirements and memory usage in resource-limited environments, making it particularly suitable for deploying large-scale models in resource-constrained scenarios, such as embedded systems or edge computing devices.

[0099] 6. Intelligent scheduling of expert models. This invention uses an intelligent scheduling module to dynamically allocate activated expert models, avoiding overuse of expert model resources and waste of computing resources. Through intelligent scheduling, the system can balance the load of activated expert models, ensuring that each expert model is not overused, while maximizing computing resources and achieving more efficient task processing.

[0100] The present invention has the following beneficial effects:

[0101] This paper proposes an automated framework based on the Mixture of Experts (MoE) architecture. This framework combines the advantages of FNNs, Transformers, and hybrid expert models, and implements efficient training and inference through a flexible gating mechanism. Compared to existing MoE technologies, this paper significantly improves on the diversity of expert models, the flexibility of the gating mechanism, and user customization capabilities, resulting in higher performance, resource efficiency, and task adaptability.

[0102] 1. Diverse selection of expert models

[0103] Limitations of existing technologies: Existing MoE model architectures (such as Switch Transformer and DeepSpeed-MoE) mainly rely on a single type of expert model (usually Transformer). Although these models perform well in natural language processing tasks, their expressiveness and flexibility are limited when dealing with non-language tasks or other specific tasks (such as structured data processing, time series analysis, etc.).

[0104] Improvements of the present invention: The present invention breaks through the limitations of a single expert model and provides a wider range of options by integrating feedforward neural networks (FNNs), transformers, and hybrid expert models. FNNs are highly efficient when processing linear tasks, transformers excel at handling complex contextual relationships, and hybrid expert models integrate the advantages of different models to handle complex data types and tasks. This diversity greatly enhances the adaptability of the model, making the present invention not only suitable for natural language processing, but also extendable to other task areas such as financial data analysis and image recognition.

[0105] 2. Flexible gating mechanism

[0106] Limitations of Existing Technologies: Many existing MoE model architectures use fixed Top-k Gating or Soft Gating mechanisms. In Top-k Gating, only the k expert models with the highest scores are selected for computation, while Soft Gating allows all expert models to participate and weight their outputs. While these methods improve computational efficiency, their fixed gating schemes can lead to insufficient performance in certain complex tasks, particularly when balancing the contributions of expert models or avoiding over-reliance on certain models.

[0107] Improvements of the present invention: This invention proposes three flexible gating mechanisms: Soft Gating, Top-k Gating, and Noisy Top-k Gating. Noisy Top-k Gating introduces random noise, making the expert model selected each time not completely fixed. This enhances the diversity of expert model selection, avoids the model's over-reliance on certain expert models, and thus improves the model's generalization ability. This flexible gating design not only optimizes the utilization of computing resources but also dynamically adjusts the expert selection strategy based on the complexity of the task, significantly improving the model's performance across different tasks.

[0108] 3. Efficient user customization and control

[0109] Limitations of existing technologies: In existing MoE model architectures, users have relatively few options for customization and control, typically limited to setting model parameter scales and training hyperparameters. Expert model selection and gating strategies are often preset, making it difficult for users to flexibly adjust them to meet different task requirements.

[0110] Improvements of the present invention: The present invention gives users more control and flexibility, allowing them to freely choose expert models (such as FNN, Transformer, or hybrid models) and gating strategies (such as Soft Gating, Top-k Gating, or Noisy Top-k Gating) during the training process. This high degree of customization allows users to adjust the model structure and training process according to the characteristics of the task, thereby achieving optimal performance in different task scenarios.

[0111] 4. Efficient use of computing resources

[0112] Limitations of existing technologies: Large-scale models typically require a large amount of computing resources and memory. Although the existing MoE model architecture reduces some of the computational workload through Top-k Gating, it still suffers from resource waste or computational bottlenecks in resource-constrained environments.

[0113] Improvements of the present invention: Through Noisy Top-k Gating, the present invention reduces computational burden while increasing model generalization and robustness. This random selection of expert models avoids repeated activation of certain fixed expert models, thereby more evenly distributing the computational burden and maximizing computing resources. Furthermore, by activating only a subset of expert models, the present invention significantly reduces memory usage, making it suitable for deployment in resource-constrained environments, such as embedded systems or edge devices.

[0114] 5. Objective improvement effect

[0115] Significantly Improved Training and Inference Efficiency: This paper introduces flexible gating mechanisms (particularly Top-k Gating and Noisy Top-k Gating) to significantly reduce the number of expert models that need to be activated during each training and inference process. Compared to traditional full-model computation, this approach of selectively activating expert models significantly reduces computational complexity, shortening model training time on large datasets while also improving inference speed.

[0116] Improving model generalization and stability: The Noisy Top-k Gating mechanism significantly improves model generalization by increasing the diversity of expert model selection, preventing the model from over-reliance on specific expert models. This improvement makes the model more stable when facing different tasks or new data, and better able to cope with changes in data distribution.

[0117] 6. Stronger task adaptability

[0118] By combining diverse expert model selection with flexible gating strategies, this invention can handle a wide range of tasks, extending beyond natural language processing to image processing, time series analysis, structured data, and other fields. Compared to existing single expert models, this invention performs better across diverse tasks.

[0119] 7. More efficient resource utilization

[0120] In resource-constrained environments, this method avoids wasting computing resources by selectively activating some expert models. This significantly reduces memory and computing resource requirements, particularly during the training and inference of large-scale models. By rationally allocating resources, this method achieves a good balance between computing efficiency and performance.

[0121] In summary, by improving the existing MoE technology, the present invention provides a more efficient, flexible and task-adaptable automation framework, which can provide users with a better training and inference experience and significantly reduce the consumption of computing resources.

[0122] The definitions of abbreviations and key terms involved in the present invention are as follows:

[0123] 1. NLP (Natural Language Processing)

[0124] NLP refers to the technology used by computers to process, understand, and generate natural languages ​​(such as human languages). NLP technology is widely used in tasks such as translation, text generation, and sentiment analysis.

[0125] 2. MoE (Mixture of Experts) Expert Model Mixture

[0126] A model architecture that improves computational efficiency and accuracy by distributing tasks among different expert models. The MoE model uses a gating mechanism to select a small number of expert models for computation, thereby improving the training and inference efficiency of large-scale models.

[0127] 3. FNN (Feedforward Neural Network)

[0128] An artificial neural network in which information is passed from the input layer through the hidden layers to the output layer without feedback loops. FNNs are widely used for tasks such as regression and classification and excel at processing linear data.

[0129] 4. Transformer

[0130] A neural network model based on the self-attention mechanism, originally designed for natural language processing tasks. Due to its parallel processing capabilities and ability to effectively capture contextual dependencies, the Transformer performs well in long sequence modeling and language tasks.

[0131] 5. Top-k Gating

[0132] An expert model selection mechanism that reduces the computational burden while ensuring model accuracy by selecting the k expert models with the highest scores to participate in the calculation.

[0133] 6. Soft Gating

[0134] Soft gating assigns weights to all expert models, ensuring that each participates in the calculation. The final output is the weighted sum of all expert models. This mechanism fully utilizes the knowledge of each expert model and is very effective when processing complex data.

[0135] 7. Noisy Top-k Gating

[0136] A mechanism that adds random noise to Top-k gating to increase the diversity of expert model selection, avoid overfitting caused by fixed selection of certain expert models, and improve the generalization ability of the model.

[0137] 8. LLM (Large Language Model)

[0138] A type of neural network model with a large number of parameters, commonly used to process natural language tasks such as text generation, translation, question answering, etc. LLM learns language patterns and context by training on large amounts of text data.

[0139] 9. Gating Mechanism

[0140] During the expert model selection process, the gating mechanism determines which expert models will be activated to participate in the calculation. The gating mechanism dynamically selects the most appropriate expert model by evaluating the characteristics of the input data.

[0141] 10. Expert Network (Expert Network / Expert Model)

[0142] Expert networks are components of the MoE model architecture, each of which specializes in handling a specific type of task. By properly selecting and combining expert networks, the model can handle complex tasks more efficiently and accurately.

[0143] 11. Deepspeed

[0144] Microsoft's open-source deep learning optimization library significantly improves the training and inference efficiency of large-scale models. Deepspeed optimizes computing resource utilization by providing mixed-precision training, parallel processing, and memory optimization technologies.

[0145] 12. vLLM

[0146] A library for optimizing large-scale language model inference, particularly excelling in reducing latency and memory consumption during inference. vLLM achieves faster responses in large-scale model inference through efficient parallelization.

[0147] 13. Routing

[0148] In the MoE model architecture, routing refers to the process of distributing input data among different expert networks. Through the routing mechanism, the model can dynamically select the most appropriate expert for processing.

[0149] 14. Overfitting

[0150] Overfitting occurs when a model overfits to the training data, causing it to perform poorly on unseen test data. Overfitting usually occurs when a model is too complex or when training for too long.

[0151] 15. Generalization

[0152] Generalization ability refers to the ability of a model to maintain good performance on new or unseen data. A model with good generalization ability can avoid overfitting and adapt to a wider range of data distributions.

[0153] 16. FLOPs (Floating Point Operations Per Second) floating point operations

[0154] A metric used to measure the computational complexity of a model, representing the number of floating-point operations performed per second. The higher the FLOPs, the greater the computational requirements of the model.

[0155] 17. Backpropagation

[0156] An algorithm used to train neural networks that adjusts weights to minimize the loss by calculating the gradient of the loss function with respect to each weight. Backpropagation is a key step in training neural networks.

Claims

1. An adaptive hybrid expert model training system for a multi-agent system, characterized by: It includes a training part, a large language model adaptive control part, and a front-end and back-end system part. The training part is used to pre-process the input data and dynamically select an expert model. The expert model is selected through a preset gating mechanism. The large language model adaptive control part is used to optimize the training part. The front-end and back-end system parts serve as the interface for user interaction with the system, providing a front-end user interaction interface and a back-end task management system module. The large language model adaptive control part includes a routing control module for dynamically adjusting the expert model activated by the MoE model; and a model monitoring and adjustment module for adaptively adjusting the parameters of the MoE model architecture and the expert model based on the monitoring results; The routing control module uses reinforcement learning or rule-based methods to dynamically adjust the expert model selection strategy at each reasoning or training step, and dynamically adjusts the expert models activated in the MoE model by monitoring the characteristics of the input data in real time; The model monitoring and adjustment module, after monitoring data is analyzed by the control algorithm, the system can automatically adjust the activation ratio, gating strategy and parameter optimization method of the expert model; the model monitoring and adjustment module is used to monitor the performance of the expert model during the training process and adaptively adjust the model architecture and parameters based on the monitoring results; The large language model adaptive control part also includes an expert scheduling module for scheduling expert models in the activated expert models; the large language model adaptive control part performs intelligent scheduling in a multi-expert model environment to ensure that the system can efficiently allocate computing resources under multiple parallel tasks.

2. The adaptive hybrid expert model training system for a multi-agent system according to claim 1, characterized in that: The training part includes a data preprocessing module, a gating mechanism module, an expert model module and a parameter optimization module. The data preprocessing module is used to preprocess the input data. The gating mechanism module dynamically selects the expert model for the preprocessed data according to the preset gating mechanism. The expert model module activates the corresponding expert model for calculation through the expert model selected by the gating mechanism module. Finally, the parameter optimization module updates the parameters of the expert model through the back propagation algorithm, optimizes the loss function and improves the performance of the expert model.

3. The adaptive hybrid expert model training system for a multi-agent system according to claim 1, characterized in that: Through the Django backend combined with statistical analysis tools, the MoE model system collects various indicators during the training process. The monitoring data is analyzed through the control algorithm, and the system automatically adjusts the activation ratio, gating strategy and parameter optimization method of the expert model. The model monitoring and adjustment module is used to monitor the performance of the expert model during the training process, including loss, accuracy, and computing resource usage, and adaptively adjust the model architecture and parameters based on the monitoring results.

4. The adaptive hybrid expert model training system for a multi-agent system according to claim 1, characterized in that: The front-end and back-end parts include a front-end user interface, a back-end task management system module, and a database and storage management module. The front-end user interface provides users with a convenient interface for model configuration, training progress viewing, and model performance analysis; The back-end task management system module supports training tasks submitted by users, and is responsible for coordinating the interaction between the front-end and the training system. It is also responsible for interacting with the database to save training records, model configurations and results; the database and storage management module is used to save training data, model parameters, training logs and results.

5. The adaptive hybrid expert model training system for a multi-agent system according to claim 1, characterized in that: The expert model can use a feedforward neural network, a Transformer model, a hybrid expert model, a convolutional neural network, a long short-term memory network, or an autoregressive model.

6. The adaptive hybrid expert model training system for a multi-agent system according to claim 1, characterized in that: The gating mechanism module adopts soft gating, top-k gating, noisy top-k gating, hard gating, evolutionary gating, and gating mechanism based on reinforcement learning.

7. The adaptive hybrid expert model training system for a multi-agent system according to claim 1, characterized in that: The large language model adaptive control part can be replaced by rule-driven adaptive control or adaptive control based on statistical analysis.

8. The adaptive hybrid expert model training system for a multi-agent system according to claim 1, characterized in that: The training part can be replaced by a meta-learning-based training strategy, a self-supervised learning training method, or mixed precision training.

Images