Enhanced AI large model engineering automatic arrangement system

Abstract

The invention relates to the field of AI, in particular to an enhanced AI large model engineering automatic arrangement system which comprises an automatic data processing module, a model development and training module, a hyper-parameter optimization module, a model evaluation and verification module, a resource management module and a monitoring and maintenance module. According to the scheme, an improved hyper-parameter optimization method is adopted to optimize hyper-parameters in AutoML, the core concept of the method is that a weight coefficient is used, the relative importance of a fitting error and a prediction error in a weighted error function is controlled, and the balance between over-fitting and under-fitting is achieved by adjusting the weight coefficient; a dynamic cloud scheduling method based on intelligent agents is adopted for resource scheduling, the method intelligently adapts to user requirements and selective service quality based on distributed learning in an edge cloud computing environment, more efficient task scheduling is achieved, and the automation level of the whole AI model life cycle is improved.

Description

Technical Field

[0001] The present invention relates to the field of AI, and in particular to an enhanced automatic orchestration system for AI large model engineering. Background Art

[0002] AI big model engineering is a comprehensive model based on advanced technologies in multiple fields such as deep learning, natural language processing, and computer vision, including AutoML (Automated machine learning), distributed computing, containerization technology, Kubernetes cluster management, and continuous integration / continuous deployment (CI / CD) processes. However, existing technologies still face challenges. AutoML may encounter overfitting or underfitting when facing extremely complex models; when performing distributed training on AI big models, there are difficulties in choosing between data parallelism and model parallelism, as well as communication bottlenecks, and data resources and server resources cannot be reasonably scheduled; in addition, as the scale of the model increases, the demand for computing power surges, resulting in high costs and difficulty in management. These are all problems that need to be solved urgently. Summary of the invention

[0003] In view of the above situation, the present invention provides an enhanced automatic orchestration system for large AI model projects. In view of the problem that AutoML may encounter overfitting or underfitting when facing extremely complex models, an improved hyperparameter optimization method is used to optimize the hyperparameters in AutoML. The core concept of this method is to use a weight coefficient 𝛼 to control the relative importance of fitting error and prediction error in the weighted error function. By adjusting the weight coefficient 𝛼, the optimal parameter combination that minimizes the weighted error function and maximizes the prediction accuracy and economic value of AutoML is found, thereby achieving a balance between overfitting and underfitting. In view of the problem that when large AI models are undergoing distributed training, there are selection difficulties between data parallelism and model parallelism and communication bottlenecks, and data resources and server resources cannot be reasonably scheduled. A dynamic cloud scheduling method based on intelligent agents is used for resource scheduling. This method intelligently adapts to user needs and selective service quality based on distributed learning in an edge cloud computing environment, and continuously optimizes its performance through hyperparameter learning, thereby achieving more efficient task scheduling in edge cloud computing scenarios and improving the automation level of the entire AI model life cycle.

[0004] An enhanced AI large model engineering automatic orchestration system provided by the present invention includes an automated data processing module, a model development and training module, a hyperparameter optimization module, a model evaluation and verification module, a resource management module, and a monitoring and maintenance module;

[0005] The automated data processing module includes data integration, data cleaning, data annotation and data enhancement to obtain standard data;

[0006] The model development and training module uses standard data sets to train large AI models and uses k GPUs and nodes for distributed training;

[0007] The hyperparameter optimization module uses an improved hyperparameter optimization method to optimize the hyperparameters in the AI ​​large model to find the best hyperparameter configuration;

[0008] The model evaluation and verification module uses cross-validation to evaluate the generalization ability of the AI ​​big model on data in actual applications, and uses A / B testing to compare the performance of different versions of the AI ​​big model;

[0009] The resource management module manages the resources of the cloud system and edge servers used by the AI ​​large model, dynamically allocates computing resources according to the task requirements of the AI ​​large model, and uses a dynamic cloud scheduling method based on intelligent agents for intelligent scheduling;

[0010] The monitoring and maintenance module continuously monitors the performance of the AI ​​big model in the production project, collects and analyzes log information during the operation of the AI ​​big model, performs fault detection and fault recovery when the AI ​​big model fails, and notifies the administrator.

[0011] Furthermore, the hyperparameter optimization module optimizes the hyperparameters of each model using an improved hyperparameter optimization method, and the improved hyperparameter optimization method specifically includes the following steps:

[0012] Step S1: define the relevant parameter range of the model, determine the model used by the AI ​​large model project and all hyperparameters to be optimized, define the candidate value range of the hyperparameters to be optimized, and obtain different combinations of hyperparameters;

[0013] Step S2: Calculate the weighted error function. For each hyperparameter combination, calculate the weighted error function and prediction error according to the defined candidate value range of the hyperparameter. The formula used is as follows: ;

[0014] in, is the number of training set samples, is the total sample size, is the weight coefficient, is the index, is the actual observed value, represents the predicted value, represents the weighted error function;

[0015] Step S3: Calculate the weighted error function of all hyperparameter combinations. For each hyperparameter combination and weight coefficient, repeat the operation of step S2 to calculate the weighted error function values ​​under all hyperparameter combinations, and form all weighted error function values ​​into a WEF matrix.

[0016] Step S4: Determine the optimal hyperparameter combination, compare all weighted error function values ​​in the WEF matrix obtained in step S3, and select the hyperparameter combination with the minimum WEF value as the optimal hyperparameter combination.

[0017] Furthermore, the resource management module uses a dynamic cloud scheduling method based on intelligent agents for intelligent scheduling, and the dynamic cloud scheduling method based on intelligent agents includes distributed learning management and intelligent multi-agent task scheduling;

[0018] Furthermore, the distributed learning management specifically includes the following steps:

[0019] Step P1: Initialization phase, obtain the environmental parameters of the cloud system and edge server, obtain the coverage according to the location and specifications of the edge server, obtain the user set and task set, and set two parameters Sumf and Meta with empty initial values;

[0020] Step P2: Process the user set, obtain historical information and current data for each user, calculate the cumulative number of violations V of the service level objective, obtain user preferences, and merge user preferences and violation number V into the parameter Meta;

[0021] Step P3: Process the task set. For each task in the task set , obtain monitoring information, calculate tasks With edge servers The affinity value between them is added to the parameter Sumf, and the parameter Sumf is merged into the parameter Meta, where is the index, and Respectively represent Tasks and edge servers;

[0022] Step P4: loop through steps P2 and P3 to update the value of parameter Meta;

[0023] Furthermore, the intelligent multi-agent task scheduling specifically includes the following steps:

[0024] Step M1: Initial task scheduling, setting thresholds for allocation and migration scheduling,

[0025] Step M2: Passive thread, Meta variables collected and aggregated from multiple agents in the cloud system and edge servers using passive threads;

[0026] Step M3: Active thread, using active thread to be responsible for initial cloud task allocation and migration scheduling;

[0027] Step M4: Conflict resolution, using deep neural networks to solve tasks Assign to edge server ;

[0028] Step M5: threshold adjustment, adjusting the thresholds of allocation and migration scheduling according to the number of edge servers and migration scheduling;

[0029] Further, step M3 specifically includes the following steps:

[0030] Step M31: Initial assignment in new tasks When arriving, find the appropriate edge server based on user preferences , the new task Assign to edge server ,in, is the index, Indicates edge servers;

[0031] Step M32: Find the edge server with the lowest parameter Sumf value If the parameter Sumf value is greater than the assigned threshold, the iteration stops. Otherwise, for each edge server Find the most suitable edge server for the task running on ;

[0032] Step M33: Task migration, if the task With edge servers The affinity between them is greater than the migration scheduling threshold, and the task Set as the migration target and schedule the migration.

[0033] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0034] (1) In view of the problem that AutoML may encounter overfitting or underfitting when facing extremely complex models, an improved hyperparameter optimization method is used to optimize the hyperparameters in AutoML. The core concept of this method is to use a weight coefficient to control the relative importance of fitting error and prediction error in the weighted error function. By adjusting the weight coefficient, the optimal parameter combination that minimizes the weighted error function and maximizes the prediction accuracy and economic value of AutoML is found, thus achieving a balance between overfitting and underfitting.

[0035] (2) In view of the difficulties in choosing between data parallelism and model parallelism and communication bottlenecks during distributed training of large AI models, and the inability to reasonably schedule data resources and server resources, a dynamic cloud scheduling method based on intelligent agents is adopted for resource scheduling. This method intelligently adapts to user needs and selective service quality based on distributed learning in the edge cloud computing environment. The dynamic cloud scheduling method based on intelligent agents consists of two logical components: distributed learning management and intelligent management of multiple agents. These two components are both independent and work together, allowing the method to autonomously adapt in a constantly changing environment and continuously optimize its performance through hyperparameter learning, thereby achieving more efficient task scheduling in edge cloud computing scenarios and improving the automation level of the entire AI model life cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of an enhanced AI large model engineering automatic arrangement system provided by the present invention;

[0037] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0039] Example 1, see Figure 1 , an enhanced AI large model engineering automatic orchestration system provided by the present invention includes an automated data processing module, a model development and training module, a hyperparameter optimization module, a model evaluation and verification module, a resource management module, and a monitoring and maintenance module;

[0040] The automated data processing module includes data integration, data cleaning, data annotation and data enhancement to obtain standard data;

[0041] The model development and training module uses standard data sets to train large AI models and uses k GPUs and nodes for distributed training;

[0042] The hyperparameter optimization module uses an improved hyperparameter optimization method to optimize the hyperparameters in the AI ​​large model, find the best hyperparameter configuration, and reduce the cost of training and reasoning;

[0043] The model evaluation and verification module uses cross-validation to evaluate the generalization ability of the AI ​​big model on data in actual applications, and uses A / B testing to compare the performance of different versions of the AI ​​big model;

[0044] The resource management module manages the resources of the cloud system and edge servers used by the AI ​​large model, dynamically allocates computing resources according to the task requirements of the AI ​​large model, and uses a dynamic cloud scheduling method based on intelligent agents for intelligent scheduling;

[0045] The monitoring and maintenance module continuously monitors the performance of the AI ​​big model in the production project, collects and analyzes log information during the operation of the AI ​​big model, performs fault detection and fault recovery when the AI ​​big model fails, and notifies the administrator.

[0046] Embodiment 2: This embodiment is based on the above embodiment. In the automated data processing module, data integration collects data from various sources for integration. Data cleaning is to identify and correct errors in the data. Data annotation is to automatically or semi-automatically add labels to the data. Data enhancement is to generate new training examples by transforming existing data.

[0047] When the monitoring and maintenance module performs fault detection, it uses heartbeat detection, timeout mechanism, redundancy check and consistency protocol to perform fault detection;

[0048] Heartbeat detection: monitors the node status by periodically sending heartbeat signals and setting a time threshold. If no heartbeat response is received from the node within the set time threshold, it is considered that the node may have a fault.

[0049] Timeout mechanism: Set a timeout for each task. If the task is not completed or does not return any status information within this time, the task is considered failed.

[0050] Redundancy check: By replicating the same data or tasks on N nodes, when one node fails, verify whether the results of other nodes are consistent and determine the failure situation.

[0051] Consensus protocol: Use a consensus algorithm to ensure that all nodes in a distributed system reach a consistent state.

[0052] Embodiment 3, based on the above embodiment, the fault recovery of the monitoring and maintenance module includes the following contents:

[0053] Checkpoint: Periodically save the computational state to persistent storage. When a failure occurs, you can recover from the most recent checkpoint, reducing the amount of redo work, but there is a trade-off between storage overhead and recovery time.

[0054] Snapshot: Similar to checkpoint, but snapshot is usually a consistent copy of the entire system state, suitable for scenarios that require global consistency;

[0055] Data redundancy: By storing data copies in L locations, data can be restored from other locations when it is lost;

[0056] Task rescheduling: For failed tasks, resubmit them to other available nodes for execution, and use the intelligent agent-based dynamic cloud scheduling method of the resource management module to dynamically allocate tasks;

[0057] State machine replication: By replicating the state machine's tasks, services can still be provided even if some nodes fail. State machine replication is usually used together with the consistency protocol.

[0058] Automatic failover: When the primary node fails, control is automatically transferred to the backup node.

[0059] Embodiment 4: This embodiment is based on the above embodiment. The hyperparameter optimization module uses an improved hyperparameter optimization method to optimize the hyperparameters of each model. The improved hyperparameter optimization method specifically includes the following steps:

[0060] Step S1: define the relevant parameter range of the model, determine the model used by the AI ​​large model project and all hyperparameters to be optimized, define the candidate value range of the hyperparameters to be optimized, and obtain different combinations of hyperparameters;

[0061] Step S2: Calculate the weighted error function. For each hyperparameter combination, calculate the weighted error function and prediction error according to the defined candidate value range of the hyperparameter. The formula used is as follows: ;

[0062] in, is the number of training set samples, is the total sample size, is the weight coefficient, is the index, is the actual observed value, represents the predicted value, represents the weighted error function;

[0063] Step S3: Calculate the weighted error function of all hyperparameter combinations. For each hyperparameter combination and weight coefficient, repeat the operation of step S2 to calculate the weighted error function values ​​under all hyperparameter combinations, and form all weighted error function values ​​into a WEF matrix.

[0064] Step S4: Determine the optimal hyperparameter combination, compare all weighted error function values ​​in the WEF matrix obtained in step S3, and select the hyperparameter combination with the minimum WEF value as the optimal hyperparameter combination.

[0065] In view of the problem that AutoML may encounter overfitting or underfitting when facing extremely complex models, an improved hyperparameter optimization method is used to optimize the hyperparameters in AutoML. The core concept of this method is to use a weight coefficient to control the relative importance of fitting error and prediction error in the weighted error function. By adjusting the weight coefficient, the optimal parameter combination that minimizes the weighted error function and maximizes the prediction accuracy and economic value of AutoML is found, thereby achieving a balance between overfitting and underfitting.

[0066] Embodiment 5, based on the above embodiment, the resource management module uses a dynamic cloud scheduling method based on intelligent agents for intelligent scheduling, and the dynamic cloud scheduling method based on intelligent agents includes distributed learning management and intelligent multi-agent task scheduling;

[0067] The distributed learning management specifically includes the following steps:

[0068] Step P1: Initialization phase, obtain the environmental parameters of the cloud system and edge server, obtain the coverage according to the location and specifications of the edge server, obtain the user set and task set, and set two parameters Sumf and Meta with empty initial values;

[0069] Step P2: Process the user set, obtain historical information and current data for each user, calculate the cumulative number of violations V of the service level objective, obtain user preferences, and merge user preferences and violation number V into the parameter Meta;

[0070] Step P3: Process the task set. For each task in the task set , obtain monitoring information, calculate tasks With edge servers The affinity value between them is added to the parameter Sumf, and the parameter Sumf is merged into the parameter Meta, where is the index, and Respectively represent Tasks and edge servers;

[0071] Step P4: loop through steps P2 and P3 to update the value of parameter Meta.

[0072] Embodiment 6, based on the above embodiment, the intelligent multi-agent task scheduling specifically includes the following steps:

[0073] Step M1: Initial task scheduling, setting thresholds for allocation and migration scheduling,

[0074] Step M2: Passive thread, Meta variables collected and aggregated from multiple agents in the cloud system and edge servers using passive threads;

[0075] Step M3: Active thread, using active thread to be responsible for initial cloud task allocation and migration scheduling;

[0076] Step M4: Conflict resolution, using deep neural networks to solve tasks Assign to edge server ;

[0077] Step M5: threshold adjustment, adjusting the thresholds of allocation and migration scheduling according to the number of edge servers and migration scheduling;

[0078] Step M3 specifically includes the following steps:

[0079] Step M31: Initial assignment in new tasks When arriving, find the appropriate edge server based on user preferences , the new task Assign to edge server ,in, is the index, Indicates edge servers;

[0080] Step M32: Find the edge server with the lowest parameter Sumf value If the parameter Sumf value is greater than the assigned threshold, the iteration stops. Otherwise, for each edge server Find the most suitable edge server for the task running on ;

[0081] Step M33: Task migration, if the task With edge servers The affinity between them is greater than the migration scheduling threshold, and the task Set as the migration target and schedule the migration.

[0082] Embodiment 7, based on the above embodiment, in step M5, the threshold value is adjusted, specifically including the following rules:

[0083] When edge server utilization is low and there are few migration tasks, increase the allocation threshold and task scheduling threshold;

[0084] When edge server utilization is high and there are many migration tasks, lower the allocation threshold and task scheduling threshold;

[0085] The allocation threshold and the task scheduling threshold are determined based on the monitoring information obtained in step P3.

[0086] In view of the difficulties in choosing between data parallelism and model parallelism and communication bottlenecks when conducting distributed training of large AI models, and the problem of inability to reasonably schedule data resources and server resources, a dynamic cloud scheduling method based on intelligent agents is adopted for resource scheduling. This method intelligently adapts to user needs and selective service quality based on distributed learning in an edge cloud computing environment. The dynamic cloud scheduling method based on intelligent agents consists of two logical components: distributed learning management and intelligent management of multiple agents. These two components are both independent and work together, enabling the method to autonomously adapt in a changing environment and continuously optimize its performance through hyperparameter learning, thereby achieving more efficient task scheduling in edge cloud computing scenarios and improving the automation level of the entire AI model life cycle.

[0087] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0088] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0089] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. An enhanced AI large model engineering automatic arrangement system, characterized by: It includes automated data processing module, model development and training module, hyperparameter optimization module, model evaluation and verification module, resource management module, and monitoring and maintenance module; The automated data processing module includes data integration, data cleaning, data annotation and data enhancement to obtain standard data; The model development and training module uses standard data sets to train large AI models and uses k GPUs and nodes for distributed training; The hyperparameter optimization module uses an improved hyperparameter optimization method to optimize the hyperparameters in the AI ​​large model to find the best hyperparameter configuration; The model evaluation and verification module uses cross-validation to evaluate the generalization ability of the AI ​​big model on data in actual applications, and uses A / B testing to compare the performance of different versions of the AI ​​big model; The resource management module manages the resources of the cloud system and edge servers used by the AI ​​large model, dynamically allocates computing resources according to the task requirements of the AI ​​large model, and uses a dynamic cloud scheduling method based on intelligent agents for intelligent scheduling; The monitoring and maintenance module continuously monitors the performance of the AI ​​big model in the production project, collects and analyzes log information during the operation of the AI ​​big model, performs fault detection and fault recovery when the AI ​​big model fails, and notifies the administrator.

2. According to claim 1, an enhanced AI large model engineering automatic arrangement system is characterized by: The hyperparameter optimization module optimizes the hyperparameters of each model using an improved hyperparameter optimization method, and the improved hyperparameter optimization method specifically includes the following steps: Step S1: define the relevant parameter range of the model, determine the model used by the AI ​​large model project and all hyperparameters to be optimized, define the candidate value range of the hyperparameters to be optimized, and obtain different combinations of hyperparameters; Step S2: Calculate the weighted error function. For each hyperparameter combination, calculate the weighted error function and prediction error according to the defined candidate value range of the hyperparameter. The formula used is as follows: ； in, is the number of training set samples, is the total sample size, is the weight coefficient, is the index, is the actual observed value, represents the predicted value, represents the weighted error function; Step S3: Calculate the weighted error function of all hyperparameter combinations. For each hyperparameter combination and weight coefficient, repeat the operation of step S2 to calculate the weighted error function values ​​under all hyperparameter combinations, and form all weighted error function values ​​into a WEF matrix. Step S4: Determine the optimal hyperparameter combination, compare all weighted error function values ​​in the WEF matrix obtained in step S3, and select the hyperparameter combination with the minimum WEF value as the optimal hyperparameter combination.

3. According to claim 1, an enhanced AI large model engineering automatic arrangement system and system, characterized in that: The resource management module uses a dynamic cloud scheduling method based on intelligent agents for intelligent scheduling, and the dynamic cloud scheduling method based on intelligent agents includes distributed learning management and intelligent multi-agent task scheduling; The distributed learning management specifically includes the following steps: Step P1: Initialization phase, obtain the environmental parameters of the cloud system and edge server, obtain the coverage according to the location and specifications of the edge server, obtain the user set and task set, and set two parameters Sumf and Meta with empty initial values; Step P2: Process the user set, obtain historical information and current data for each user, calculate the cumulative number of violations V of the service level objective, obtain user preferences, and merge user preferences and violation number V into the parameter Meta; Step P3: Process the task set. For each task in the task set , obtain monitoring information, calculate tasks With edge servers The affinity value between them is added to the parameter Sumf, and the parameter Sumf is merged into the parameter Meta, where is the index, and Respectively represent Tasks and edge servers; Step P4: loop through steps P2 and P3 to update the value of parameter Meta.

4. According to claim 3, an enhanced AI large model engineering automatic arrangement system and system are characterized by: The intelligent multi-agent task scheduling specifically includes the following steps: Step M1: Initial task scheduling, setting thresholds for allocation and migration scheduling, Step M2: Passive thread, Meta variables collected and aggregated from multiple agents in the cloud system and edge servers using passive threads; Step M3: Active thread, using active thread to be responsible for initial cloud task allocation and migration scheduling; Step M4: Conflict resolution, using deep neural networks to solve tasks Assign to edge server ; Step M5: threshold adjustment, adjusting the thresholds of allocation and migration scheduling according to the number of edge servers and migration scheduling; Step M3 specifically includes the following steps: Step M31: Initial assignment in new tasks When arriving, find the appropriate edge server based on user preferences , the new task Assign to edge server ,in, is the index, Indicates edge servers; Step M32: Find the edge server with the lowest parameter Sumf value If the parameter Sumf value is greater than the assigned threshold, the iteration stops. Otherwise, for each edge server Find the most suitable edge server for the task running on ; Step M33: Task migration, if the task With edge servers The affinity between them is greater than the migration scheduling threshold, and the task Set as the migration target and schedule the migration.

Images