Self-adaptive hybrid expert model training framework of multi-agent system

Abstract

The invention provides a self-adaptive hybrid expert model training framework of a multi-agent system, which comprises a training part, a large language model self-adaptive regulation and control part and a front-end and rear-end system part, and is characterized in that the training part is used for preprocessing input data and dynamically selecting an expert model; the expert model performs selection through a preset gating mechanism, the large language model adaptive regulation and control part is used for optimizing the training part, and the front-end and rear-end system part serves as an interface for interaction between a user and the system and provides a front-end user interaction interface and a rear-end task management system module. The adaptive hybrid expert model training framework of the multi-agent system shows significant advantages in the aspects of expert model diversity, flexible gating mechanism, user-defined ability and real-time training feedback.

Description

Technical Field

[0001] The present invention relates to a hybrid expert model, and in particular to an adaptive hybrid expert model training framework for a multi-agent system. Background Art

[0002] In the current field of multi-agent systems and artificial intelligence (AI) technology, deep learning models, especially large language models (LLMs), have made significant progress in many fields such as natural language processing (NLP), computer vision, and reinforcement learning. These models rely on huge parameter scales and a large amount of computing resources to capture complex patterns and structures. However, as the scale of model parameters increases, the cost of training and inference also increases exponentially. This poses new challenges for how to effectively train and deploy these models.

[0003] In the existing technology, the Mixture of Experts (MoE) model has gradually become an effective solution, which aims to reduce the computational burden and memory usage by selectively activating some expert networks in the model, thereby improving the training efficiency and reasoning speed of the model. The MoE model introduces a gating mechanism to dynamically select the activated expert model according to the characteristics of the input data, avoiding the computational redundancy of the entire model. Typical MoE models such as Google's SwitchTransformer and Microsoft's DeepSpeed-MoE have significantly improved the performance of large-scale models through this mechanism.

[0004] However, the existing MoE model architecture also faces some limitations:

[0005] (1) Single choice of expert model: Most existing MoE models are limited to using Transformer experts and lack a variety of expert model choices, making them inflexible in handling different task types.

[0006] (2) Fixed gating mechanism: Many MoE models use fixed Top-k Gating or Soft Gating. Although these mechanisms improve computational efficiency to a certain extent, they are insufficient in terms of model generalization ability and ability to cope with complex tasks.

[0007] (3) Limited user customization capabilities: In existing MoE models, user customization options are relatively limited and usually rely on preset optimization schemes, making it difficult to flexibly adjust the model architecture and training strategies according to different tasks or requirements.

[0008] With the development of AI technology, more and more application scenarios require not only powerful model performance, but also efficient use of computing resources and the ability to adapt to different types of tasks. For example, in a large-scale multimodal system, structured data, unstructured text, and image processing may be involved at the same time, which places higher demands on the flexibility and customization of the model. Traditional MoE models show certain limitations in dealing with these diverse needs. Summary of the invention

[0009] The present invention provides an adaptive hybrid expert model training framework for a multi-agent system. By improving the existing MoE model architecture, a more flexible and efficient solution is provided, which can better adapt to diverse task requirements and provide efficient training and reasoning support in an environment with limited computing resources. It is used to solve the problem of providing users with a highly customizable MoE model training and reasoning platform by combining different types of expert models (such as FNN, Transformer, hybrid expert model) and multiple gating mechanisms (such as Soft Gating, Top-kGating, Noisy Top-k Gating), thereby improving the performance and generalization ability of the model. The technical solution is as follows:

[0010] An adaptive hybrid expert model training framework for a multi-agent system includes a training part, a large language model adaptive regulation part, and a front-end and back-end system part. The training part is used to preprocess input data and dynamically select an expert model, and the expert model is selected through a preset gating mechanism. The large language model adaptive regulation part is used to optimize the training part. The front-end and back-end system parts serve as interfaces for user interaction with the system, providing a front-end user interaction interface and a back-end task management system module.

[0011] The training part includes a data preprocessing module, a gating mechanism module, an expert model module and a parameter optimization module. The data preprocessing module is used to preprocess the input data. The gating mechanism module dynamically selects the expert model for the preprocessed data according to the preset gating mechanism. The expert model module activates the corresponding expert model for calculation through the expert model selected by the gating mechanism module. Finally, the parameter optimization module updates the parameters of the expert model through the back propagation algorithm, optimizes the loss function and improves the performance of the expert model.

[0012] The large language model adaptive control part includes a routing control module for dynamically adjusting the expert model activated by the MoE model; and a model monitoring and adjustment module for adaptively adjusting the parameters of the MoE model architecture and the expert model according to the monitoring results.

[0013] The large language model adaptive control part also includes an expert scheduling module for scheduling expert models in the activated expert models.

[0014] Through the Django background combined with statistical analysis tools, the MoE model system collects various indicators during the training process, and the monitoring data is analyzed through the control algorithm. The system automatically adjusts the activation ratio, gating strategy and parameter optimization method of the expert model. The model monitoring and adjustment module is used to monitor the performance of the expert model during the training process, including loss, accuracy, and computing resource usage, and adaptively adjusts the model architecture and parameters according to the monitoring results.

[0015] The front-end and back-end parts include a front-end user interface, a back-end task management system module, and a database and storage management module. The front-end user interface provides users with a convenient interface for model configuration, training progress viewing, and model performance analysis; the back-end task management system module supports training tasks submitted by users, and is responsible for coordinating the interaction between the front-end and the training system, and is also responsible for interacting with the database to save training records, model configurations, and results; the database and storage management module is used to save training data, model parameters, training logs, and results.

[0016] The expert model can use a feedforward neural network, a Transformer model, a hybrid expert model, a convolutional neural network, a long short-term memory network, or an autoregressive model.

[0017] The gating mechanism module adopts Soft Gating, Top-k Gating, Noisy Top-k Gating, hard gating, evolutionary gating, and gating mechanism based on reinforcement learning.

[0018] The large language model adaptive control part can be replaced by rule-driven adaptive control or adaptive control based on statistical analysis.

[0019] The training part can be replaced by a training strategy based on meta-learning, a training method based on self-supervised learning, or mixed precision training.

[0020] The adaptive hybrid expert model training framework of the multi-agent system is an innovative solution based on the Mixture of Experts (MoE) model architecture, providing a flexible, efficient, and customizable large-scale model training and reasoning platform. Compared with the prior art, the present invention has significant advantages in terms of expert model diversity, flexible gating mechanism, user customization capability, and real-time training feedback. Through the analysis of the technical solution of the present invention, the following technical effects and advantages can be obtained, and the reasons for their occurrence can be further explained through reasonable reasoning.

[0021] (1) Expert model diversity improves task adaptability

[0022] Limitations of existing technologies: Current MoE model architectures are usually limited to a single type of expert model, such as Transformer, which is limited when dealing with complex or diverse tasks. These models are less effective when dealing with non-language tasks (such as structured data or time series analysis), resulting in their lack of adaptability. Among them, the expert network / expert model refers to the E in MoE, that is, multiple expert models constitute a hybrid expert model.

[0023] Improvements and effects of the present invention: The present invention integrates multiple types of expert models, including feedforward neural networks (FNNs), transformers, and hybrid expert models, so that the system of the MoE model can flexibly select different expert models according to the characteristics of the task. FNNs are good at processing linear tasks, while transformers are good at complex sequence tasks. Hybrid expert models can comprehensively utilize the advantages of different models to deal with complex data. The system is a training, deployment, and deployment system for the MoE model.

[0024] This diverse selection of expert models makes the system not only suitable for natural language processing (NLP) tasks, but also expands to other application scenarios, such as financial data analysis, image processing, time series data prediction, etc. By providing more expert model choices, the system can better cope with different types of tasks, significantly improving the adaptability of tasks and the expressiveness of the model.

[0025] (2) Flexible gating mechanism improves computational efficiency and model generalization capability

[0026] Limitations of existing technologies: Most of the gating mechanisms of existing MoE model architectures use fixed strategies, such as Top-kGating or Soft Gating. Although these fixed gating strategies have been optimized in terms of computing resources, they are insufficient in terms of model generalization and handling of diverse tasks.

[0027] Improvements and effects of the present invention: The present invention designs a flexible gating mechanism, including Soft Gating, Top-k Gating and Noisy Top-k Gating. In particular, the Noisy Top-k Gating mechanism introduces randomness, so that the expert model activated each time is not completely fixed, which increases the diversity of expert model selection, thereby improving the generalization ability of the model and avoiding over-reliance on specific expert models.

[0028] This flexible gating mechanism enables the system to improve the generalization ability of the model without sacrificing computational efficiency, especially in scenarios with uneven data distribution or high task complexity. The introduction of random noise enables the MoE model to avoid overfitting certain expert models, enhancing its stability and robustness in different task scenarios. In addition, through the combination of Top-k Gating and Noisy Top-k Gating, the system effectively utilizes computing resources while reducing the computational burden, achieving a balance between computational efficiency and generalization ability.

[0029] (3) User customization capabilities improve system flexibility and controllability

[0030] Limitations of existing technologies: Traditional MoE model architectures usually provide users with limited customization options, making it difficult for users to flexibly adjust expert models and gating strategies according to task requirements. This results in a lack of fine-grained control over the model when the system handles different tasks, and the model's potential cannot be fully utilized.

[0031] Improvements and effects of the present invention: By opening up the user's right to choose expert models and gating mechanisms, the user can flexibly configure the model architecture and training process according to specific tasks. The user can not only choose different expert models, but also freely choose a suitable gating mechanism, such as Soft Gating, Top-k Gating or Noisy Top-k Gating.

[0032] This customization capability enables the system to better adapt to complex or special task requirements. For example, users can choose FNN models when processing efficient linear tasks, Transformer when processing language tasks, and hybrid expert models when processing multimodal data. At the same time, users can dynamically adjust the gating strategy according to the limitations of computing resources and the generalization requirements of the task, thereby maximizing the flexibility and controllability of the system. This highly customized design greatly improves the applicability and user experience of the system in different application scenarios.

[0033] (4) Real-time analysis and training feedback of large language models improve training efficiency and effectiveness

[0034] Limitations of existing technologies: In the traditional MoE model training process, model training feedback is usually processed offline or in batches, and it is impossible to analyze and adjust the model performance in real time during the training process. This approach may lead to low training efficiency or overfitting problems.

[0035] Improvements and effects of the present invention: By integrating a large language model (LLM), the system can analyze the performance of the model in real time during the training process and provide feedback and suggestions based on the analysis results. The system can dynamically adjust training parameters such as learning rate, expert model selection strategy or the number of activated expert models according to the current loss curve, expert model selection and resource usage, thereby optimizing the training process.

[0036] Through real-time analysis of large language models, the system can automatically identify model deficiencies during training and make timely adjustments in the middle stages of training. This real-time feedback mechanism enables the system to correct overfitting, bias, or unreasonable resource allocation problems at an early stage, avoiding wasting training time and computing resources. In addition, by dynamically adjusting the parameters of the model, the system can find the optimal expert model selection strategy and gating mechanism at different stages, thereby improving training efficiency and ensuring that the model can achieve optimal performance with the least possible resource consumption.

[0037] (5) Resource utilization efficiency has been greatly improved

[0038] Limitations of existing technologies: The training and inference of large-scale models usually require huge computing resources and memory usage, especially in resource-constrained environments (such as embedded devices or edge computing scenarios), where traditional MoE models are difficult to deploy efficiently.

[0039] Improvements and effects of the present invention: The present invention reduces the number of expert models activated in each training or reasoning process through Top-k Gating and Noisy Top-k Gating, thereby reducing the system's computation and memory consumption. At the same time, through the introduction of the adaptive control module, the system can dynamically adjust the allocation of computing resources according to the current task requirements, avoiding unnecessary waste of resources.

[0040] By reducing the number of activations of the expert model, the system significantly reduces the computational requirements during training and inference. Especially when dealing with large-scale models, this selective activation strategy can significantly reduce memory usage and computational burden. The adaptive control module further ensures that the system can be efficiently deployed in environments with limited computing resources, while avoiding excessive resource consumption by allocating resources reasonably. This design greatly improves the resource utilization efficiency of the system, making it more suitable for deployment in different computing environments.

[0041] In summary, the present invention has shown significant advantages over the existing MoE architecture through various technical improvements. By introducing diversified expert models, flexible gating mechanisms, user customization capabilities, and real-time training feedback of large language models, the system can provide better task adaptability, higher training efficiency, and stronger generalization capabilities based on efficient use of computing resources. These technical effects effectively solve key problems in the prior art through reasonable design and innovative implementation methods, and improve the practical application performance of large-scale models in training and reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the adaptive hybrid expert model training framework for the multi-agent system. DETAILED DESCRIPTION

[0043] like Figure 1 As shown in the figure, the adaptive hybrid expert model training framework of the multi-agent system includes a training part, a large language model adaptive regulation part, and a front-end and back-end system part. The training part is used to pre-process the input data and dynamically select the expert model, and the expert model is selected through a preset gating mechanism; the large language model adaptive regulation part is used to optimize the training part; the front-end and back-end system part serves as an interface for the user to interact with the MoE model system, providing a front-end user interaction interface and a back-end task management system module. Each part plays a key role in the workflow of the entire system, from the pre-processing of training data, the dynamic adjustment of the expert model to the presentation of the user interaction interface, ensuring an efficient model training and reasoning process.

[0044] 1. Training part: The training part is the core functional part of the present invention, which is responsible for the preprocessing of input data, the selection of expert models and the optimization of model parameters. It includes a data preprocessing module, a gating mechanism module, an expert model module and a parameter optimization module. The data preprocessing module is used to preprocess the input data. The gating mechanism module dynamically selects the expert model for the preprocessed data according to the preset gating mechanism. The expert model module activates the corresponding expert model for calculation through the expert model selected by the gating mechanism module. Finally, the parameter optimization module updates the parameters of the expert model through the back propagation algorithm, optimizes the loss function and improves the performance of the expert model.

[0045] The above modules are specifically described as follows:

[0046] 1. Data preprocessing module, which preprocesses the input data, including text cleaning, data enhancement, feature extraction, etc. The preprocessed data will be passed to the gating mechanism module as preprocessed data.

[0047] Preprocessing can utilize Python's multi-threaded processing and data processing libraries such as Pandas, combined with natural language processing techniques (such as Tokenization, word embedding, etc.) to preprocess input data.

[0048] 2. The gating mechanism module calculates the contribution value of each expert model through a gating network, which can usually be implemented using a smaller neural network. In Soft Gating, all expert models participate in the calculation, and the final output is a weighted sum; in Top-k Gating, only the k expert models with the highest scores are selected to participate; NoisyTop-k Gating introduces noise to make the expert model selection random, increasing the diversity and generalization ability of the model.

[0049] The gating mechanism module dynamically selects an expert model for preprocessed data according to a preset gating mechanism (Soft Gating, Top-k Gating or NoisyTop-k Gating), and different gating mechanisms correspond to different computational burdens and generalization capabilities.

[0050] 3 Expert model modules, each expert model is designed independently. FNN is good at processing linear tasks, Transformer is suitable for complex sequence tasks, and the hybrid expert model can comprehensively utilize the advantages of different networks. The training part uses PyTorch or DeepSpeed ​​framework for parallel acceleration and optimization.

[0051] The expert model module activates the corresponding expert model (FNN, Transformer or hybrid expert model) for calculation according to the result selected by the gating mechanism module. Each expert model is optimized for a specific task type or data feature to improve computational efficiency and model performance.

[0052] 4. Parameter optimization module, based on the automatic differentiation method, combined with the optimization algorithm (such as Adam or SGD), realizes the automatic optimization of the parameters of the expert model. The update of the parameters is limited to the activated expert model, thereby reducing unnecessary consumption of computing resources.

[0053] In the training part, the parameter optimization module updates the parameters of the expert model through the back-propagation algorithm, optimizes the loss function and improves the performance of the expert model.

[0054] The second large language model adaptive control part: The large language model adaptive control part is the intelligent control part of the present invention, which is responsible for adaptively optimizing the MoE model architecture and expert model through training and reasoning to ensure that the system can dynamically adjust the model to achieve optimal performance in different task scenarios. It includes a routing control module and a model monitoring and adjustment module, which are respectively used to dynamically adjust the expert model activated by the MoE model, and adaptively adjust the parameters of the MoE model architecture and expert model according to the monitoring results. In addition, it also includes an expert scheduling module for reasonable scheduling of expert models in the activated expert models.

[0055] 1. Routing control module, which uses reinforcement learning or rule-based methods to dynamically adjust the expert model selection strategy at each reasoning or training step. In specific implementations, the routing network will combine current input features and historical performance to select the expert model that can provide the best performance.

[0056] The routing control module dynamically adjusts the expert models activated in the MoE model by monitoring the characteristics of the input data in real time. By adaptively adjusting different expert models, the system of the MoE model can select the optimal expert model combination according to different input conditions.

[0057] 2. Model monitoring and adjustment module. Through the Django background combined with statistical analysis tools, the system can collect various indicators during the training process. After the monitoring data is analyzed by the control algorithm (such as Bayesian optimization or genetic algorithm), the system can automatically adjust the activation ratio, gating strategy and parameter optimization method of the expert model.

[0058] The model monitoring and adjustment module is used to monitor the performance of the expert model during the training process (such as loss, accuracy, computing resource usage, etc.), and adaptively adjust the model architecture and parameters based on the monitoring results. The system dynamically changes the gating strategy and expert model selection according to task requirements.

[0059] 3 Expert scheduling module, based on the scheduling algorithm (such as polling scheduling or priority-based scheduling algorithm), the system reasonably schedules the activated expert model. DeepSpeed ​​or other distributed training libraries are used to optimize the scheduling of computing tasks under multi-GPU / multi-node.

[0060] This module is responsible for intelligent scheduling in a multi-expert model environment, ensuring that the system can efficiently allocate computing resources under multiple parallel tasks and avoid excessive use of expert models or waste of computing resources.

[0061] 3. Front-end and back-end parts: The front-end and back-end parts are interfaces for users to interact with the system, responsible for providing an intuitive and easy-to-use user interface, and supporting the scheduling, monitoring and visualization of the training process in the background. It includes the front-end user interface, the back-end task management system module, and the database and storage management module.

[0062] 1. Front-end user interface. Use Javascript (such as React or Vue) to develop the front-end interface, combined with data visualization libraries such as D3.js or Plotly.js to display key data in the training process, such as loss curves, expert model selection ratios, etc.

[0063] The front-end user interface provides users with a convenient interface for model configuration, training progress viewing, and model performance analysis. Users can select training tasks, configure expert models and gating mechanisms, and view the training progress and performance indicators of the MoE model through a graphical interface.

[0064] 2. The backend task management system module is built on the Django framework. The backend is responsible for receiving model configuration requests from users and managing the scheduling of model training tasks through task queues (such as Celery). Django interacts with the PyTorch / DeepSpeed ​​library to control the training process.

[0065] The backend task management system module supports training tasks submitted by users and is responsible for coordinating the interaction between the front end and the training system; it is also responsible for interacting with the database to save training records, model configurations and results.

[0066] 3. Database and storage management module, which can connect to PostgreSQL or other relational databases for data storage, and combine with Django ORM for data management and query. The intermediate results of the model and the final model weights are stored and managed using a distributed file system.

[0067] The database and storage management module can save training data, model parameters, training logs and results, ensuring that users can access training history and results at any time.

[0068] Furthermore, the present invention may use other alternatives for the expert model, the gating mechanism, the replacement of the adaptive control part, the training part, and the front-end and back-end parts.

[0069] 1. Alternatives to Expert Models

[0070] Original solution: This invention uses three types of expert models - feedforward neural network (FNN), Transformer and hybrid expert model, which are optimized for linear tasks, complex context tasks and multimodal tasks respectively. Alternative solutions can use convolutional neural network (CNN), long short-term memory network (LSTM), autoregressive model as expert model.

[0071] As an expert model, convolutional neural network (CNN) is a very efficient model architecture in image processing tasks or other tasks that need to capture spatial features. Therefore, CNN can be used as an expert model to replace FNN or Transformer, especially for computer vision tasks and image classification tasks. By introducing CNN, the present invention can be further extended to multimodal data processing, including the combination of different data sources such as images, text and audio.

[0072] Long short-term memory network (LSTM) as an expert model: For time series analysis or tasks that require processing long sequence dependencies, LSTM or its variants (such as GRU) can replace FNN or Transformer models as expert models that specialize in processing time series data. This alternative can improve the performance of the model in financial data prediction, sensor data analysis, etc.

[0073] Autoregressive models as expert models: In some language tasks, autoregressive models (such as GPT variants) can be introduced to replace the standard Transformer model. Autoregressive models are suitable for tasks that require the gradual generation of prediction values, especially in text generation and machine translation.

[0074] 2. Alternatives to gating mechanisms

[0075] Original solution: This paper proposes three gating mechanisms, including Soft Gating, Top-k Gating and NoisyTop-k Gating, to optimize the selection of expert models and the utilization of computing resources. Alternative solutions can use hard gating, gating mechanism based on reinforcement learning, and evolutionary gating as gating mechanisms.

[0076] Hard Gating: Unlike Soft Gating, the hard gating mechanism only activates the expert model with the highest score and ignores other expert models. Although hard gating may reduce generalization ability in some tasks, it can greatly reduce the computational burden and is particularly suitable for task scenarios with high requirements for inference speed. This alternative can be used in extremely resource-constrained or low-power devices.

[0077] Gating mechanism based on reinforcement learning: In the alternative, the gating mechanism can be dynamically adjusted through reinforcement learning strategies. The system can adaptively select the most appropriate expert model by learning the requirements of the task in real time. This method can further optimize the selection strategy of the expert model and make iterative adjustments based on the model performance and feedback information, thereby improving the accuracy and flexibility of the expert model selection.

[0078] Evolutionary Gating: By using evolutionary algorithms (such as genetic algorithms) to select expert models, the system can gradually optimize model performance by "evolutionary" selecting expert model combinations. Each selected expert model combination can be adjusted based on previous performance to ensure continuous model improvement.

[0079] 3. Alternative solutions for adaptive regulation

[0080] Original solution: The present invention uses a large language model to analyze and regulate the training process in real time, and provides an automatic adjustment mechanism based on model feedback. Alternative solutions can use rule-driven adaptive regulation and adaptive regulation based on statistical analysis as the adaptive regulation part.

[0081] Rule-driven adaptive control: An alternative to large language model analysis is to implement adaptive control through a predefined rule system. For example, users can define different expert model selection strategies and gating mechanisms according to different stages of the task. The system dynamically adjusts the training process according to preset conditions (such as the convergence speed of the loss function and resource usage) without relying on real-time analysis of the large language model.

[0082] Adaptive control based on statistical analysis: The system can use statistical analysis tools to evaluate the performance of the model in real time and adjust the training parameters of the model based on simple statistical indicators (such as average error, variance, etc.). This method can reduce the complexity of implementation while still being able to dynamically control the training process.

[0083] 4. Alternatives to the training part

[0084] Original solution: In the training part, the model parameters are updated through traditional back-propagation algorithms (such as Adam, SGD, etc.), and the distributed computing framework (such as DeepSpeed) is combined to accelerate the training. Alternative solutions can use training strategies based on meta-learning, self-supervised learning training methods, and mixed precision training as the training part.

[0085] Meta-learning-based training strategy: The system can use meta-learning algorithms to quickly adjust the model's parameters when the task changes, thereby accelerating the model's adaptability. Meta-learning can help the model quickly learn the optimal strategy with a small number of samples. This alternative is particularly suitable for online learning or incremental learning tasks.

[0086] Self-supervised learning training method: Through self-supervised learning methods, the system can be trained without labeled data, using the information of the input data itself to generate learning objectives. This method can effectively improve model performance when there is insufficient labeled data, and does not rely on a large number of manually labeled data sets.

[0087] Mixed precision training: Mixed precision training technology can further accelerate the training process of large-scale models. By converting some computing tasks to half-precision floating point numbers (FP16), the system can reduce the computing burden and improve training efficiency while ensuring model accuracy.

[0088] 5. Alternative solutions for front-end and back-end

[0089] Original solution: The front end of the present invention uses a user interface based on JavaScript (such as React, Vue), and the back end uses the Django framework to interact with the task management system, and combines the distributed file system for data and model storage. Alternative solutions can use other types of front-end alternative solutions, back-end alternative solutions, and storage alternative solutions.

[0090] Front-end alternatives: The front-end user interface can be implemented using other frameworks, such as Angular or Svelte. At the same time, the user interface can also be provided through a command line interface (CLI) or a simplified mobile application. In some usage scenarios, a simple CLI is more suitable for specific developers or operators than a graphical interface.

[0091] Backend alternatives: The backend can use other frameworks instead of Django, such as Flask, FastAPI, or Node.js. Different backend frameworks can adapt to different scalability and performance requirements. At the same time, the task scheduling system can replace the existing Celery and use other message queues such as Apache Kafka or RabbitMQ to support more complex parallel processing and task scheduling.

[0092] Storage alternatives: Other file systems or cloud storage solutions can be used for storage. For example, alternative storage methods to HDFS or S3 can be Google Cloud Storage, Azure Blob Storage, etc. In addition, for local deployment, the system can also use a local database (such as SQLite) to store training logs and model data, further simplifying the complexity of the system.

[0093] The present invention has the following characteristics:

[0094] 1. Diversified expert model selection mechanism. The present invention integrates FNN (feedforward neural network), Transformer and hybrid expert model into the MoE model architecture, breaking the limitation of single expert model in the prior art and allowing the system to select the most suitable expert model according to task requirements. By providing more diverse expert model selection, the present invention significantly improves the system's task adaptability and ability to process multimodal data, and expands the application of the MoE model architecture in different fields.

[0095] 2. Flexible gating mechanism design. The present invention uses three gating mechanisms, including Soft Gating, Top-kGating and Noisy Top-k Gating, allowing users to flexibly choose according to task requirements, and introduces randomness through Noisy Top-kGating to improve the generalization ability of the model. Compared with the existing fixed gating mechanism, the flexible gating design of the present invention not only improves the computational efficiency of the model, but also enhances the generalization ability of the model by introducing randomness, reduces the risk of overfitting, and has stronger adaptability.

[0096] 3. Real-time adaptive control module. The present invention integrates a large language model (LLM) or a rule-based adaptive control module, which can monitor and analyze the training process in real time, and automatically adjust the selection of expert models, gating strategies and training parameters according to feedback. Through real-time monitoring and automatic adjustment, the present invention significantly improves the efficiency and effect of the training process, avoids the delay and resource waste caused by manual adjustment in traditional systems, and enables the model to quickly adapt to task changes and ensure optimal performance.

[0097] 4. User customization and control capabilities. The present invention provides a high degree of user customization capabilities, allowing users to independently select expert models and gating mechanisms, adjust model structures and training strategies, and optimize model performance for different task scenarios. Compared with the prior art, the user customization capabilities of the present invention greatly improve the flexibility of the system. Users can dynamically adjust the model according to task requirements, making it suitable for a variety of application scenarios from small-scale tasks to large-scale complex tasks.

[0098] 5. Efficient computing resource management and utilization. The present invention reduces the number of expert models activated in each training and reasoning through Top-k Gating and Noisy Top-k Gating, and introduces a dynamic resource management mechanism to ensure efficient utilization of computing resources. The present invention can significantly reduce computing requirements and memory usage in resource-limited environments, and is particularly suitable for deploying large-scale models in resource-limited scenarios, such as embedded systems or edge computing devices.

[0099] 6. Intelligent scheduling mechanism of expert models. The present invention dynamically allocates activated expert models through an intelligent scheduling module, avoiding excessive use of expert model resources and waste of computing resources. Through intelligent scheduling, the system can balance the load of activated expert models, ensuring that each expert model is not overused, while maximizing the use of computing resources to achieve more efficient task processing.

[0100] The present invention has the following beneficial effects:

[0101] This paper proposes an automation framework based on the Mixture of Experts (MoE) architecture, which combines the advantages of FNN, Transformer and mixed expert models, and realizes efficient training and reasoning processes through a flexible gating mechanism. Compared with the existing MoE technology, this paper has made significant improvements in the diversity of expert models, the flexibility of gating mechanisms and user customization capabilities, thereby bringing higher performance, resource utilization efficiency and task adaptability.

[0102] 1. Diverse selection of expert models

[0103] Limitations of existing technologies: Existing MoE model architectures (such as Switch Transformer and DeepSpeed-MoE) mainly rely on a single expert model type (usually Transformer). Although these models perform well in natural language processing tasks, their expressiveness and flexibility are limited when dealing with non-language tasks or other specific tasks (such as structured data processing, time series analysis, etc.).

[0104] Improvements of the present invention: The present invention breaks through the limitations of a single expert model and provides a wider range of choices by integrating feedforward neural networks (FNN), Transformers, and hybrid expert models. FNN is efficient in processing linear tasks, Transformers are good at processing complex contextual relationships, and hybrid expert models integrate the advantages of different models and can handle complex data types and tasks. This diversity greatly enhances the adaptability of the model, making the present invention not only suitable for natural language processing, but also expandable to other task areas, such as financial data analysis, image recognition, etc.

[0105] 2. Flexible gating mechanism

[0106] Limitations of existing technologies: Many existing MoE model architectures use fixed Top-k Gating or SoftGating mechanisms. In Top-k Gating, only the k expert models with the highest scores are selected to participate in the calculation, while Soft Gating allows all expert models to participate and weight the output. Although these methods improve computational efficiency, the fixed gating method can easily lead to insufficient performance in some complex tasks, especially when it is necessary to balance the contribution of expert models or avoid over-reliance on certain expert models.

[0107] Improvements of the present invention: The present invention proposes three flexible gating mechanisms, namely Soft Gating, Top-k Gating and Noisy Top-k Gating. Among them, Noisy Top-k Gating introduces random noise, so that the expert model selected each time is not completely fixed, which enhances the diversity of expert model selection and avoids the model's excessive dependence on certain expert models, thereby improving the generalization ability of the model. This flexible gating design can not only optimize the utilization of computing resources, but also dynamically adjust the expert selection strategy according to the complexity of the task, thereby significantly improving the performance of the model in different tasks.

[0108] 3. Efficient user customization and control

[0109] Limitations of existing technologies: In the existing MoE model architecture, users have relatively few options to customize and control, usually limited to setting the model's parameter scale, training hyperparameters, etc. The selection and gating strategies of expert models are often preset, making it difficult for users to flexibly adjust them according to different task requirements.

[0110] Improvements of the present invention: The present invention gives users more control and flexibility, allowing users to freely choose expert models (such as FNN, Transformer or hybrid models) and gating strategies (such as Soft Gating, Top-k Gating or Noisy Top-k Gating) during the training process. This high degree of customization enables users to adjust the structure and training process of the model according to the characteristics of the task, thereby achieving optimal performance in different task scenarios.

[0111] 4. Efficient use of computing resources

[0112] Limitations of existing technologies: Large-scale models usually require a lot of computing resources and memory. Although the existing MoE model architecture reduces some of the computational workload through Top-k Gating, there are still problems of resource waste or computing bottlenecks in resource-constrained environments.

[0113] Improvements of the present invention: Through Noisy Top-k Gating, the present invention can increase the generalization ability and robustness of the model while reducing the computational burden. This mechanism of randomly selecting expert models avoids repeated activation of certain fixed expert models, thereby distributing the computational burden more evenly and maximizing the use of computing resources. In addition, by activating only some expert models, the present invention significantly reduces memory usage and is suitable for deployment in resource-constrained environments, such as embedded systems or edge devices.

[0114] 5. Objective improvement effect

[0115] Significantly improve training and reasoning efficiency This paper introduces a flexible gating mechanism (especially Top-k Gating and Noisy Top-k Gating) to significantly reduce the number of expert models that need to be activated during each training and reasoning process. Compared with the traditional full-model calculation method, this selective activation of expert models significantly reduces the computational complexity, shortens the training time of the model on large-scale data sets, and improves the reasoning speed.

[0116] Improve the generalization ability and stability of the model. The Noisy Top-k Gating mechanism avoids the model's over-reliance on certain specific expert models by increasing the diversity of expert model selection, significantly improving the generalization ability of the model. This improvement makes the model more stable when facing different tasks or new data, and can better cope with changes in data distribution.

[0117] 6. Stronger task adaptability

[0118] Through diverse expert model selection and flexible gating strategies, the present invention can handle various types of tasks, not only limited to natural language processing, but also extended to image processing, time series analysis, structured data and other fields. Compared with the existing single expert model, the present invention performs better when facing diversified tasks.

[0119] 7. Higher efficiency in resource utilization

[0120] In a resource-limited environment, the present invention avoids the waste of computing resources by selectively activating some expert models, especially in the training and reasoning process of large-scale models, which can significantly reduce the demand for memory and computing resources. By reasonably allocating resources, the present invention achieves a good balance between computing efficiency and performance.

[0121] In summary, the present invention provides a more efficient, flexible and task-adaptable automation framework by improving the existing MoE technology, which can provide users with a better training and reasoning experience and significantly reduce the consumption of computing resources.

[0122] The definitions of abbreviations and key terms used in the present invention are as follows:

[0123] 1. NLP (Natural Language Processing)

[0124] Refers to the technology of computer processing, understanding and generating natural language (such as human language). NLP technology is widely used in tasks such as translation, text generation, and sentiment analysis.

[0125] 2. MoE (Mixture of Experts) Expert model mixture

[0126] A model architecture that improves computational efficiency and accuracy by assigning tasks to different expert models. The MoE model selects a small number of expert models for computation through a gating mechanism, thereby improving the training and inference efficiency of large-scale models.

[0127] 3. FNN (Feedforward Neural Network)

[0128] An artificial neural network in which information is passed from the input layer through the hidden layers to the output layer without a feedback loop. FNNs are widely used for tasks such as regression and classification and are good at processing linear data.

[0129] 4. Transformer

[0130] A neural network model based on the self-attention mechanism, originally used for natural language processing tasks. Due to its parallel processing capabilities and the ability to effectively capture contextual dependencies, Transformer performs well in long sequence modeling and language tasks.

[0131] 5. Top-k Gating

[0132] An expert model selection mechanism reduces the computational burden while ensuring the accuracy of the model by selecting the k expert models with the highest scores to participate in the calculation.

[0133] 6. Soft Gating

[0134] Soft gating assigns weights to all expert models, so that each expert model participates in the calculation, and the final output is the weighted result of all expert models. This mechanism can make full use of the knowledge of each expert model and is very effective in processing complex data.

[0135] 7. Noisy Top-k Gating

[0136] A mechanism that adds random noise based on Top-k gating to increase the diversity of expert model selection, avoid overfitting caused by fixed selection of certain expert models, and improve the generalization ability of the model.

[0137] 8. LLM (Large Language Model)

[0138] A type of neural network model with a large number of parameters, usually used to handle natural language tasks such as text generation, translation, question answering, etc. LLM learns language patterns and context by training on large amounts of text data.

[0139] 9. Gating Mechanism

[0140] In the expert model selection process, the gating mechanism determines which expert models will be activated to participate in the calculation. The gating mechanism dynamically selects the most appropriate expert model by evaluating the characteristics of the input data.

[0141] 10. Expert Network

[0142] Expert networks are an integral part of the MoE model architecture, and each expert network focuses on handling a specific type of task. By properly selecting and combining expert networks, the model can handle complex tasks more efficiently and accurately.

[0143] 11. Deepspeed

[0144] Microsoft's open source deep learning optimization library can significantly improve the training and reasoning efficiency of large-scale models. Deepspeed optimizes the use of computing resources by providing mixed precision training, parallel processing, memory optimization and other technologies.

[0145] 12. vLLM

[0146] A library for optimizing large-scale language model inference, especially in reducing latency and memory consumption during inference. vLLM can achieve faster response in large-scale model inference through efficient parallelization mechanism.

[0147] 13. Routing

[0148] In the MoE model architecture, routing refers to the process of distributing input data among different expert networks. Through the routing mechanism, the model can dynamically select the most appropriate expert for processing.

[0149] 14. Overfitting

[0150] Overfitting means that the model is overfitted to the training data, causing it to perform poorly on unseen test data. Overfitting usually occurs because the model is too complex or the training time is too long.

[0151] 15. Generalization

[0152] Generalization ability refers to the ability of a model to maintain good performance on new or unseen data. A model with good generalization ability can avoid overfitting and adapt to a wider data distribution.

[0153] 16. FLOPs (Floating Point Operations Per Second) floating point operations

[0154] An indicator used to measure the computational complexity of a model, indicating the number of floating-point operations performed per second. The higher the FLOPs, the greater the computational requirements of the model.

[0155] 17. Backpropagation

[0156] An algorithm for training neural networks that adjusts weights to minimize the loss by calculating the gradient of the loss function with respect to each weight. Backpropagation is a key step in training neural networks.

Claims

1. An adaptive hybrid expert model training framework for multi-agent systems, characterized by: It includes a training part, a large language model adaptive control part, and a front-end and back-end system part. The training part is used to preprocess the input data and dynamically select an expert model. The expert model is selected through a preset gating mechanism. The large language model adaptive control part is used to optimize the training part. The front-end and back-end system parts serve as an interface for user interaction with the system, providing a front-end user interaction interface and a back-end task management system module.

2. The adaptive hybrid expert model training framework for a multi-agent system according to claim 1, characterized in that: The training part includes a data preprocessing module, a gating mechanism module, an expert model module and a parameter optimization module. The data preprocessing module is used to preprocess the input data. The gating mechanism module dynamically selects the expert model for the preprocessed data according to the preset gating mechanism. The expert model module activates the corresponding expert model for calculation through the expert model selected by the gating mechanism module. Finally, the parameter optimization module updates the parameters of the expert model through the back propagation algorithm, optimizes the loss function and improves the performance of the expert model.

3. The adaptive hybrid expert model training framework for a multi-agent system according to claim 1, characterized in that: The large language model adaptive control part includes a routing control module for dynamically adjusting the expert model activated by the MoE model; and a model monitoring and adjustment module for adaptively adjusting the parameters of the MoE model architecture and the expert model according to the monitoring results.

4. The adaptive hybrid expert model training framework for a multi-agent system according to claim 3, characterized in that: The large language model adaptive control part also includes an expert scheduling module for scheduling expert models in the activated expert models.

5. The adaptive hybrid expert model training framework for a multi-agent system according to claim 3, characterized in that: Through the Django background combined with statistical analysis tools, the MoE model system collects various indicators during the training process, and the monitoring data is analyzed through the control algorithm. The system automatically adjusts the activation ratio, gating strategy and parameter optimization method of the expert model. The model monitoring and adjustment module is used to monitor the performance of the expert model during the training process, including loss, accuracy, and computing resource usage, and adaptively adjusts the model architecture and parameters according to the monitoring results.

6. The adaptive hybrid expert model training framework for a multi-agent system according to claim 1, characterized in that: The front-end and back-end parts include a front-end user interface, a back-end task management system module, and a database and storage management module. The front-end user interface provides users with a convenient interface for model configuration, training progress viewing, and model performance analysis; The back-end task management system module supports training tasks submitted by users, and is responsible for coordinating the interaction between the front-end and the training system. It is also responsible for interacting with the database to save training records, model configurations and results; the database and storage management module is used to save training data, model parameters, training logs and results.

7. The adaptive hybrid expert model training framework for a multi-agent system according to claim 1, characterized in that: The expert model can use a feedforward neural network, a Transformer model, a hybrid expert model, a convolutional neural network, a long short-term memory network, or an autoregressive model.

8. The adaptive hybrid expert model training framework for a multi-agent system according to claim 2, characterized in that: The gating mechanism module adopts Soft Gating, Top-k Gating, Noisy Top-k Gating, hard gating, evolutionary gating, and gating mechanism based on reinforcement learning.

9. The adaptive hybrid expert model training framework for a multi-agent system according to claim 1, characterized in that: The large language model adaptive control part can be replaced by rule-driven adaptive control or adaptive control based on statistical analysis.

10. The adaptive hybrid expert model training framework for a multi-agent system according to claim 1, characterized in that: The training part can be replaced by a training strategy based on meta-learning, a training method based on self-supervised learning, or mixed precision training.

Images