Method for automating deployment of artificial intelligence models

Abstract

The present application provides a method for automatically deploying an artificial intelligence model, which simplifies the model building process through systematic data preprocessing, model selection, parameter optimization and performance monitoring mechanism, dynamically updates or switches the model in the application environment, maintains the overall prediction performance at the best state, and improves the performance of the model in the multi-application environment.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for automating the deployment of artificial intelligence models. Background Technology

[0002] In today's digital enterprise operating environment, more and more organizations are trying to use artificial intelligence (AI) or machine learning (ML) technologies to assist in decision-making, optimize processes, and improve overall efficiency. In these fields, the typical machine learning model development process often involves data collection, feature engineering, and model selection before finally deploying the model to real-world operational scenarios. However, existing technologies generally face the following problems.

[0003] First, the deployment process is often overly cumbersome. Most enterprises or organizations must repeatedly migrate and adjust between the development environment and the actual application environment when importing AI models. This process is easily limited by incomplete integration of data sources and system architecture, making it both time-consuming and prone to errors. Furthermore, when the amount of existing data reaches tens or even hundreds of millions of records, traditional artificial intelligence or machine learning models cannot be effectively built without proper data preprocessing. On the other hand, when external conditions or data distribution change, the previously deployed models often lose accuracy or stability. Current technologies typically rely on personnel to recollect data, repeatedly test algorithms, and manually optimize settings to maintain model performance; such a highly manual operation and maintenance method is not only time-consuming and labor-intensive but also makes it difficult to respond to business needs in real time.

[0004] Furthermore, insufficient model interpretability is a major obstacle. After deployment, traditional AI models often struggle to clearly present their predictive logic or key features to users or decision-makers. In-depth investigation of the sources or causes of model prediction biases often requires complex additional tools or experimental analysis, increasing communication costs and raising the barrier to entry. This makes it even more difficult for operations managers without a professional background to effectively control and utilize the model results.

[0005] Furthermore, while most existing systems can monitor the performance of model predictions, they are mostly limited to passive detection. Once a model performance degradation is observed, manual evaluation is usually required to determine whether to replace the model or conduct a new round of training, rather than automatically comparing it with other feasible candidate models or quickly initiating optimization mechanisms. The lack of such automated capabilities for dynamic updates and replacements makes it difficult to adjust overall operational efficiency and prediction quality in real time as the data environment changes.

[0006] In manufacturing settings, factories and enterprises often face limitations due to the budget and maintenance costs of existing measurement solutions when attempting in-depth analysis of energy consumption and efficiency across various equipment or production line units. For example, most traditional factories or offices rely on a single large electricity meter to track total electricity consumption, failing to break it down to individual machines or production units. Installing independent meters for accurate measurement requires additional hardware, installation, and maintenance resources for each unit or machine, resulting in high initial costs and long-term management burdens. Since these additional investments are often difficult to recoup, they frequently deter companies from pursuing more refined data collection and model applications.

[0007] Therefore, how to improve the adaptability of AI models to different data sources and automatically monitor and optimize AI models deployed in actual production or operation scenarios through intelligent data analysis and model management mechanisms, without significantly adding or modifying hardware, has become a crucial issue that existing technologies urgently need to address. The aforementioned problems illustrate the challenges currently faced by the industry in deploying and maintaining AI models, and also highlight the necessity of quickly switching or optimizing models while maintaining reasonable costs. These shortcomings in technology and application are precisely the motivation that has driven the further development and improvement of this invention. Summary of the Invention

[0008] The main objective of this invention is to provide a method for automating the deployment of artificial intelligence models. This method can be executed in one or more systems. Through systematic data preprocessing, model selection, and parameter optimization, it can not only simplify the model building process, but also dynamically update or switch models in the application environment, so as to maintain the overall prediction performance at its best and improve the performance of the model in diverse application environments.

[0009] To achieve the aforementioned objectives, this invention provides a method for automatically deploying artificial intelligence models, comprising: receiving operational data related to the operation or performance of at least one entity system, the operational data originating from at least one data source; preprocessing the received operational data, including but not limited to lossless data compression, outlier cleaning, and missing value processing, to generate a structured dataset required for modeling; selecting at least one candidate algorithm based on at least one data feature in the structured dataset; constructing multiple artificial intelligence models, each with multiple hyperparameter combinations, according to the selected candidate algorithm; optimizing the parameters of the multiple artificial intelligence models; evaluating the performance metrics of each artificial intelligence model and generating corresponding model interpretation results; selecting an optimized artificial intelligence model based on the performance metrics and deploying it in the application environment corresponding to the at least one data source.

[0010] After deploying the artificial intelligence model to the application environment, the method further includes the following steps: monitoring the performance index of the artificial intelligence model based on changes in real-time data generated from at least one data source in the application environment; and automatically optimizing the artificial intelligence model when the performance index is lower than a predetermined standard.

[0011] The optimization of the artificial intelligence model includes the following steps: preprocessing the newly added real-time data in the application environment; based on the newly added real-time data, if the amount of real-time data is huge, performing lossless data compression, then cleaning up outliers and handling missing values, updating the weight of the artificial intelligence model in judging the importance of features, and screening features that have a significant impact on model performance to readjust the hyperparameters of the artificial intelligence model and / or the selected feature set; retraining the artificial intelligence model to generate a retrained artificial intelligence model; and redeploying the retrained artificial intelligence model to the application environment.

[0012] After deploying the AI ​​model to the application environment, the method further includes the following steps: monitoring the performance metric of the AI ​​model based on changes in real-time data generated from at least one data source in the application environment. Further, when the performance metric falls below a predetermined standard, the following steps are performed: retrieving at least one previously built but not deployed AI model that failed to be deployed; comparing the model interpretation results corresponding to each failed AI model with the change state of the real-time data; selecting the failed AI model that best matches the change state of the real-time data; and deploying the failed AI model to the application environment to replace the existing AI model.

[0013] The step of selecting the at least one candidate algorithm based on the at least one data feature in the structured dataset further includes the following steps: selecting at least one candidate algorithm from the algorithm library based on the at least one data feature; training the at least one candidate algorithm using a random portion of data from the structured dataset and evaluating it according to at least one performance metric; and selecting the at least one candidate algorithm that performs well on the at least one performance metric.

[0014] The optimization of the parameters for the multiple artificial intelligence models further includes the following steps: adjusting the multiple hyperparameters in each artificial intelligence model, including but not limited to learning rate, regularization coefficient, model structure parameters, and batch size; and performing optimization based on a preset parameter tuning strategy, which is a grid search, random search, or heuristic optimization method.

[0015] Specifically, for each different parameter combination of the artificial intelligence model, comparisons are made based on at least one performance indicator, and the parameter combination with superior performance is selected as the final parameter setting for the artificial intelligence model.

[0016] Furthermore, the model's explanation result is generated based on the prediction results and internal parameters after training for each parameter combination.

[0017] The evaluation of the performance metrics of multiple AI models and the generation of corresponding model interpretation results further includes the following steps: For each AI model, based on its prediction result and internal parameters, calculate the global contribution of at least one data feature to the AI ​​model, where the prediction result is the overall prediction result for the structured dataset; Combining the global contribution result, calculate the local impact value of the model prediction result of at least one data feature for a single data instance or a representative subset selected from the structured dataset; Based on the calculation results of the global contribution and the local impact value, simulate the corresponding model output value for different feature values ​​of at least one data feature, and calculate the range of variation in the model prediction result or classification probability caused by the change in the feature value.

[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) Fully automated model building and deployment: This method can automatically complete the preprocessing, algorithm selection and model optimization processes for the input operational data, greatly simplifying the tediousness of manual intervention, shortening the model building cycle and reducing the overall development and maintenance costs.

[0019] (2) Dynamic monitoring and automatic update mechanism: By continuously monitoring the performance of the model deployed in the application environment, once a decline in model performance or a significant change in data distribution is detected, the system can automatically trigger the optimization or retraining process, thereby quickly adjusting the model parameters and features to ensure that the model is maintained in the best state in real time.

[0020] (3) Multi-model comparison and replacement: Compared with the traditional approach of only iterating on a single model, this method retains and manages multiple candidate models (including previously built but not yet launched models that were rejected). When the performance of the existing model deteriorates, it can automatically compare the explanatory information of these candidate models with the new data distribution, quickly select a more suitable model to launch, shorten the decision time and avoid the lengthy retraining process.

[0021] (4) Parallel global and local interpretation: While evaluating the model performance, this method also analyzes the global contribution of each feature and the local impact value of a single instance, helping users understand the model's decision-making logic, identify important features that affect prediction, and further simulate the prediction differences caused by changes in the values ​​of each feature, thereby improving the model's transparency and interpretability.

[0022] (5) Flexible adaptation to diverse data sources: Through this automated deployment process and continuous optimization mechanism, the system can process operational data from different data sources simultaneously or in turn. Even if an enterprise can only use simplified measurement devices or a single unified measurement (such as a large electricity meter recording various energy consumption), the artificial intelligence model constructed by this method can still be dynamically updated and adjusted, reducing the threshold for hardware installation and maintenance.

[0023] In summary, this method emphasizes an automated, dynamic, and interpretable AI model management process. It not only enables rapid deployment of the initial model but also automatically monitors, replaces, or optimizes it during subsequent maintenance. Simultaneously, it provides users with explanations of the model's internal decision-making processes, addressing the shortcomings of existing technologies in terms of deployment efficiency, maintenance difficulty, and interpretability. Through these improved mechanisms, the feasibility and economic benefits of implementing AI solutions in diverse application scenarios for enterprises or organizations can be effectively enhanced.

[0024] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0025] Figure 1 This is a main flowchart of the method for automatically deploying artificial intelligence models provided in an embodiment of the present invention; Figure 2 A flowchart illustrating the first dynamic optimization of the method for automatically deploying an artificial intelligence model provided in an embodiment of the present invention; Figure 3 This is a flowchart of the second dynamic optimization of the method for automatically deploying artificial intelligence models provided in an embodiment of the present invention. Attached Figure

[0026] S101, S102, S103, S104, S105, S106, S107: Steps; S201, S202, S203, S204, S205, S206, S207, S208: Steps; S301, S302, S303, S304, S305, S306, S307: Steps. Detailed Implementation

[0027] Embodiments of the present invention will be further described below with reference to the accompanying drawings. Wherever possible, the same reference numerals represent the same or similar components in the drawings and description. It is understood that components not specifically shown or described in the drawings or description are of a form known to those skilled in the art. Those skilled in the art can make various changes and modifications based on the content of this invention.

[0028] like Figure 1 A method for automatically deploying an artificial intelligence model according to an embodiment includes steps S101 to S107, as detailed below.

[0029] Step S101: Receive operational data related to the operation or performance of at least one entity system. This operational data may include historical operational data, historical performance records, or other data related to the system's operating status, such as: operating status data of machinery and equipment, electricity consumption or energy consumption data, or product production efficiency or quality data.

[0030] In step S101, the system provides a data management interface and related functions for importing and managing operational data from different sources, including but not limited to CSV files, relational databases, time-series databases, or No-SQL databases.

[0031] Systems using this method can also provide "add / edit / delete data items" functions, allowing users to create multiple data items based on their actual applications.

[0032] A specific example of step S101 can be: (1) Obtain hourly recorded electricity consumption (kWh) and peak electricity consumption, as well as actual production quantity, scrap quantity, and production line utilization rate for each shift from two data sources: the factory's total electricity meter and the production line ERP (Enterprise Resource Planning) system.

[0033] (2) Create a data project called “Factory Energy Consumption Analysis”.

[0034] (3) Upload the total electricity consumption records for the past three months to this project through the API / database link provided by the system, and define the fields of the records (time, electricity consumption, device number, etc.).

[0035] (4) Import the production volume corresponding to each production line from the ERP system.

[0036] (5) Check and clean up null values ​​in the system (if the electricity consumption record for a certain period is lost, mark it and you can choose to interpolate or remove it), and eliminate extreme abnormal values.

[0037] (6) Finally, output a structured table (e.g., a CSV file or database table) containing fields such as "timestamp, electricity consumption, production line ID, output, and product type". Step S102: Preprocess the received operational data to generate a structured dataset required for modeling. The purpose of this step is to further preprocess the previously managed and screened operational data to generate a structured dataset that can be used for model building.

[0038] In this step, the system can read the fields and number of records from the specified source based on the data items selected by the user in step S101, and remove unnecessary or duplicate data fields through filtering and sorting mechanisms. Next, the system provides visualization functions for multivariate or univariate data, allowing users to observe the data distribution across different time periods, product types, or machines through charts. During this process, if null values ​​or abnormal values ​​are detected, the system will prompt the user to select a handling method such as "remove," "replace with a specified value," or "mark as Missing" to ensure that subsequent training is not affected by noise. For example, in a factory application, if the electricity consumption during a certain period is found to be significantly higher than the normal range, it may indicate a reading error, and the user can discard the data or make reasonable adjustments according to the actual situation. Furthermore, if data from different sources differ in time series, this embodiment also supports unifying the time zone or time format and ensures that each data point corresponds to the same time segment through an alignment mechanism.

[0039] To assist users in data exploration, the system generates a "Data Exploration Report" based on the filtered results. This report includes basic statistics (such as mean, standard deviation, maximum / minimum values) and simple visualizations (such as scatter plots, line graphs, correlation coefficient heatmaps, etc.), allowing users to easily assess data quality. Finally, the system outputs the cleaned dataset in a standardized format (such as CSV, DataFrame, or other structured files), which can be directly used in subsequent steps. This preprocessing effectively reduces noise and inconsistencies in the original data, improving the success rate and accuracy of model training.

[0040] Step S103: Select at least one candidate algorithm based on at least one data feature in the structured dataset.

[0041] In this step, the structured dataset generated in steps S101-S102 is first examined, including its field attributes (such as whether the target variable is continuous or categorical) and data distribution. Next, suitable candidate algorithms, such as linear regression, random forest, or support vector machines, are selected based on user needs or automation mechanisms. If the data is large-scale and the labels are unclear, unsupervised or semi-supervised models can also be selected. For greater customization, the system allows the use of a "Customized Algorithm" to integrate user-defined functions or algorithm logic.

[0042] For example, if the analysis focuses on predicting a factory's future electricity consumption, and the target field (electricity consumption value) is continuous data, the system can present various regression algorithms (such as generalized linear regression, random forest regression, extreme gradient boosting, etc.) in the candidate list. If the factory wants to identify shifts with "abnormal power consumption," the system can provide classification or anomaly detection algorithms (such as isolation forest or univariate support vector machine). Finally, based on the selected algorithm, the system will further proceed to subsequent steps such as "hyperparameter tuning" and "model evaluation" to complete a complete model development process. Therefore, this invention can effectively select the algorithm most suitable for the target data characteristics and application scenario under automated or semi-automated guidance.

[0043] Optionally, step S103 can also be implemented by maintaining an "algorithm library" internally within the system. This library can contain various types of machine learning or deep learning algorithms, such as linear regression, random forest, extreme gradient boosting, support vector machines, and convolutional neural networks. In factory energy consumption prediction applications, if the data characteristics are continuous electricity consumption and production records, the system will initially screen for "regression" algorithms; if it is anomaly detection, it will lock onto algorithms with "outlier detection" capabilities. For example: (1) Regression: Generalized linear regression, Glmnet, random forest regression, extreme gradient regression, convolutional neural network regression; (2) Classification: Logistic Regression, Random Forest, Extreme Gradient Classification, Support Vector Machine, Convolutional Neural Network Classifier; (3) Anomaly detection (unlabeled or partially labeled data): Principal component analysis, isolated forest, univariate support vector machine, etc.

[0044] To compare multiple candidate algorithms in a short time, the system compresses data from a structured dataset to create a "fast test set." This compressed dataset represents the overall distribution characteristics while having lower training costs. Taking factory electricity consumption analysis as an example, data from only the past week or a portion of the production lines, totaling hundreds of thousands of records, can be used for preliminary training and testing of various algorithms through data compression technology, shortening the experimental cycle.

[0045] For each shortlisted candidate algorithm, the system performs short-term training using the aforementioned compressed data and calculates corresponding performance metrics (such as RMSE and MAE for regression models, or Accuracy and F1-score for classification models). Based on the test results of each algorithm, the system selects one or more algorithms that perform best on the relevant metrics and records them as "final candidate algorithms." Once the final candidate algorithms are determined, the system can further perform formal model training on a complete or large-scale dataset, applying automated parameter tuning strategies (such as grid search and random search) to obtain the optimal parameter combination, and then incorporate it into subsequent stages such as "model performance evaluation" and "model interpretation."

[0046] Step S104: Construct multiple artificial intelligence models based on at least one selected candidate algorithm.

[0047] In this step, based on the candidate algorithms selected in step S103 (such as generalized linear regression, random forest, extreme gradient boosting, etc.), one or more artificial intelligence model versions are automatically or semi-automatically created for each algorithm. Specific methods include, but are not limited to, the following processes: (1) Initial construction of multiple models: For each selected algorithm, the system uses different initial parameters or random seeds to train multiple sets of models. Taking factory electricity consumption prediction as an example, 15 models can be generated for "random forest", and each model adopts a different number of trees or sample sampling strategy when it starts; at the same time, multiple versions will be generated for "linear regression" model using different regularization coefficients.

[0048] (2) Customizable model program or external algorithm: If the user needs to customize the algorithm logic, the system can load external program code or specific mathematical functions through "customized algorithm". In this case, multiple customized models can be generated based on different initial parameters or hyperparameter configurations to expand the adaptability to various data distributions.

[0049] (3) Temporarily store and wait for subsequent parameter adjustment: After the initial generation of these models, they can undergo only basic training and validation, and the results can be stored in the system. The system will also record information such as the algorithm name, initial parameters, and training set performance of each model.

[0050] For example, in a factory electricity consumption forecasting scenario, step S104 can generate multiple model versions with different algorithms or configurations at once, such as "random forest regression," "generalized linear regression (with L1 regularization)," and "XGBoost regression." Users or the system itself can then compare the performance and stability of each model in subsequent tests and ultimately select the best-performing combination for deployment.

[0051] Step S105: Optimize the parameters of the multiple artificial intelligence models.

[0052] The implementation methods in this step include, but are not limited to, the following processes: (1) Initial evaluation of multiple models: Since multiple artificial intelligence models have been constructed in step S104, each model uses the same or different algorithms and initial parameter settings. Therefore, this step will first perform a basic evaluation of these models and collect indicators such as loss function value, accuracy, RMSE, etc., as the basis for subsequent parameter tuning.

[0053] (2) Automated hyperparameter tuning strategy: Then, following the preset hyperparameter tuning strategy (such as grid search, random search, or Bayesian optimization heuristics), different hyperparameter combinations are tried one by one. For example, in the scenario of factory power consumption forecasting, if the random forest algorithm is used, the system will test different numbers of trees, maximum depth, number of tree nodes, etc.; if linear regression is used, its regularization coefficient, learning rate, etc. are adjusted. Through multiple training and cross-validation, parameter combinations that perform well on certain indicators are found.

[0054] (3) Cross-validation (k-fold cross-validation) and performance comparison: To improve the reliability of parameter tuning, cross-validation can be performed for each parameter combination, and metrics such as mean error or accuracy can be calculated. The performance of each combination on the test set will be recorded for subsequent aggregation and comparison. For classification problems, performance can also be measured using metrics such as F1-score and Precision / Recall. For unlabeled or semi-labeled problems of anomaly detection, performance can be measured using metrics such as TPR (True Positive Rate), Contamination, and Min-Distance.

[0055] (4) Parameter optimization results: The system will eventually select the combination that performs best in terms of comprehensive indicators (such as RMSE, Accuracy or other customized indicators) from all the parameter combinations tried, and mark this combination as the "final setting". At this point, the optimal parameters for each model have been determined and can be used for further evaluation and interpretation in the next stage.

[0056] Optionally, step S105 can also be predefined by the system as a set of "hyperparameter search ranges," covering learning rate, regularization factor, model structure parameters (such as tree depth, number of neurons, etc.), or batch size. For example, in random forests, the system can perform grid search for "number of trees (N_estimators)" and "maximum depth (Max_depth)"; in XGBoost, it can adjust both "learning rate" and "L2 regularization (l2_reg)." Optional hyperparameter tuning strategies include grid search, random search, or Bayesian optimization (a heuristic method), determined by the system or the user. If the data scale is large and the parameter space is wide, random search or Bayesian optimization can also be selected to improve efficiency. The system automatically generates multiple hyperparameter combinations based on the above "search range" and "hyperparameter tuning strategies," and performs multiple training and testing cycles on each model. During the process, the system will record information such as training time, memory usage, and performance metrics for each parameter combination.

[0057] After the training process described above is completed, the system will collect the main performance metrics (such as RMSE, MAE, MAPE, Accuracy, F1-score, etc., depending on the application type) for each parameter combination. For the same model, the system can plot the results of "parameter combination vs. performance metrics" and automatically select the parameter combination that performs best in that metric, setting it as the "final parameter settings". For example, if the random forest has the lowest RMSE when the number of trees is 200 and the maximum depth is 10, this combination will be regarded as the final hyperparameters of the model; similarly, if the extreme hyperparameter enhancement model has the best performance when the learning rate is 0.05 and the regularization coefficient is 1.0, it will also be recorded as the final settings.

[0058] In addition to quantifying and comparing performance metrics, the system also generates corresponding explanatory results for the model trained for each parameter combination. For example, it uses SHAP (Shapley Value) or Feature Importance to display the importance of each feature to the prediction results. It can also compare the changes in feature importance under different parameter combinations to identify why certain parameter settings are particularly sensitive to specific features. Finally, after training is complete, the system can output a comparison table or visual report of "parameter combinations, performance metrics, and feature contributions" to help users or managers better understand the model's operating logic and key rationale under different settings.

[0059] Once the system has completed training and interpretation generation for all parameter combinations, it will select the combination that is optimal in terms of metrics and presents reasonable interpretation results based on the aforementioned performance comparison, and use it as the "final parameter setting" for the model. If multiple models are being tuned simultaneously (e.g., Random Forest vs. Limit Improvement), each model can select its optimal solution, allowing subsequent steps (e.g., step S106) to further evaluate its performance and interpretability, and then select one or more of the best models for deployment.

[0060] Step S106: Evaluate the performance metrics of each AI model and generate corresponding model explanation results.

[0061] The implementation methods in this step, particularly the methods for generating performance metrics and model interpretation results, include, but are not limited to, the following processes: (1) Performance Evaluation (Model Evaluation Index): After completing the parameter tuning in step S105, the system can execute testing or verification procedures for each artificial intelligence model. For example, in regression scenarios, metrics such as RMSE, MAE, and MAPE can be calculated; for classification problems, Accuracy, Precision, Recall, F1-score, and even a confusion matrix can be plotted to observe the accuracy of each category. Taking factory electricity consumption prediction as an example, the system will output various error metrics based on the difference between the actual electricity consumption value and the model's predicted value, allowing users to judge the model's prediction effect.

[0062] (2) Visualization Reports (Model Evaluation Plots): To facilitate understanding of the model's prediction performance on test data, the system can automatically generate corresponding charts. For example, "actual value vs. predicted value scatter plot" and "residual distribution plot" for regression problems, or "ROC curve" and "PR curve" for classification problems. These visualization tools allow users to more intuitively distinguish which models are closer to the real data in their predictions and which models can maintain stability under abnormal conditions.

[0063] (3) Model Explanation: The system simultaneously generates explanation results for each model, which may include techniques such as "Partial Dependence Plot (PDP)" or "Shapley Value" to reveal the contribution of key features in model prediction. Taking factory applications as an example, if it is observed that "shift" and "temperature" have high weights in electricity consumption prediction, users can further adjust scheduling or capacity planning; in classification scenarios, SHAP analysis can be used to analyze which feature is most likely to lead to a certain type of judgment, which helps to discover potential problems (such as equipment failure tendency).

[0064] Optionally, after completing the training and basic performance evaluation of multiple artificial intelligence models (such as RMSE, Accuracy, F1-score, etc.), the system can also enter a more refined interpretation process to help users gain a deeper understanding of "which features play a key role in the model's prediction results" and "how specific input feature values ​​affect the final prediction." The implementation methods in this optional embodiment include, but are not limited to, the following processes: (1) Calculating Global Contribution: The system calculates the global contribution of each artificial intelligence model (e.g., random forest regression, extreme degree enhancement regression, etc.) based on its training results and internal parameters (e.g., tree structure, weight vector). Taking factory electricity consumption forecasting as an example, the system can determine the influence of features such as "shift", "daily temperature", and "previous period electricity consumption" in the global forecast by averaging the native FeatureImportance and SHAP values ​​of each algorithm, or by using a custom partial dependency analysis. For example, the system can determine that the average importance of "temperature" is 0.35 and "shift" is 0.25, indicating that "temperature" has the highest weight in the overall forecast performance.

[0065] (2) Calculating Local Impact Values: Next, the system selects a single data instance (such as the electricity consumption record of a specific production line on a certain day and shift) or a small, representative subset of data (such as a random sample of the night shift production line in the current month) to examine the impact of each feature on the prediction of that instance at the "Local" level. For example, if the "temperature" feature has a particularly high value in a certain data point, and the model has a significant positive impact on this feature, the contribution to the final electricity consumption prediction value can be calculated to be as high as +8 kWh; if "shift" or "cumulative output" only has a small impact on that instance, the contribution can be relatively low. The system can accurately quantify the contribution of each feature to the individual prediction results through record-by-record Shapley Value, LIME (Local Interpretable Model-agnostic Explanations), or Partial Dependence Analysis.

[0066] (3) Simulating the variation range of different feature values: Based on the information of "global contribution" and "local influence value" mentioned above, the system further simulates "how the model output will change if the feature value changes". Taking the factory power consumption scenario as an example, if we want to explore the sensitivity of "temperature" to the prediction results, we can gradually increase or decrease the "temperature" between 15°C and 35°C and observe the corresponding change in the predicted power consumption. For example, the system can calculate that every 1°C increase will increase the predicted power consumption by 3kWh to 5kWh. In classification scenarios (such as judging whether a shift belongs to "high energy consumption anomaly"), the system can present results such as "if the feature value changes, the probability of the shift being classified as "abnormal" increases from 30% to 70%", so that managers can make more accurate scheduling or maintenance judgments.

[0067] This implementation reveals the model's prediction logic simultaneously in both global and local dimensions. Furthermore, it allows for simulations under different feature values, calculating the range of changes in prediction results or classification probabilities. This helps users not only understand "which features are most important" but also accurately grasp "how adjusting a feature value will affect the final model output in a specific context." This integrated explanation mechanism significantly benefits AI applications requiring high reliability and transparency, such as factory production, medical diagnosis, and financial risk control.

[0068] Step S107: Select an optimized AI model based on the performance metric and deploy it in the application environment corresponding to at least one data source. This step is used to bring the optimized and evaluated best model online to assist in processing data from a specific data source.

[0069] In this step, all models that have undergone parameter optimization and performance evaluation are first aggregated, and the best-performing model is selected automatically or by the user. Next, the system packages the model into an executable file, container, or service and deploys it to a specified application environment, such as a factory management system, cloud platform, or local server. If the model needs to process data in real time, it can integrate daily inputs from data sources (such as sensor data and ERP records) and continuously provide predictions or judgments; if it involves batch analysis, model calculations can be triggered at specific times. Taking factory electricity applications as an example, once deployed, the system will input current electricity consumption and production data into the model in real time or periodically, outputting predictions of future electricity demand or abnormal situations, helping decision-makers manage production lines more efficiently.

[0070] like Figure 2 According to another embodiment, the method includes providing a process for dynamically monitoring the performance of an artificial intelligence model configured in an application environment and optimizing the model after performing step S107, including steps S201 to S208, as detailed below.

[0071] Step S201: Based on changes in real-time data generated from at least one data source in the application environment, monitor the performance metrics of the artificial intelligence model.

[0072] Step S202: Check if the performance indicators of the artificial intelligence model are lower than the predetermined standard. If not, proceed to step S203; if yes, proceed to step S204.

[0073] In steps S201-S202, after the artificial intelligence model is formally deployed and running in the application environment, the system continuously monitors the model's performance metrics (such as RMSE, Accuracy, or F1-score) to ensure that the model's prediction results remain at the expected level. If the system detects that the model's performance begins to decline and the metric value is lower than the threshold set by the user or the system in advance (i.e., the predetermined standard described in step S202), the retraining procedure can be automatically triggered. If the system is configured to require human confirmation, the administrator will be notified so that they can manually select "Retraining" to start the process (i.e., begin executing steps S204-S208).

[0074] Step S203: Maintain the original AI model deployed in the application environment, and return to step S201 to continuously monitor the performance of the AI ​​model.

[0075] Step S204: Optimize the artificial intelligence model automatically.

[0076] Step S205: Preprocess the newly added real-time data in the application environment.

[0077] In this step, the system imports real-time data accumulated since the last training and performs necessary preprocessing, such as correcting field formats, handling null and outlier values, and merging the original structured dataset. Taking factory electricity monitoring as an example, this new data may include the latest meter readings, shift information, and production capacity records.

[0078] Step S206: Based on the newly added real-time data, if the amount of real-time data is huge, perform lossless data compression, then perform outlier cleaning and missing value processing, then update the weight of the features judged by the artificial intelligence model, and screen features that have a significant impact on the model performance, so as to readjust the hyperparameters of the artificial intelligence model and / or the selected feature set.

[0079] In this step, by comparing the feature distributions of the new data with those of the old data, the system can dynamically adjust the importance of features (e.g., through SHAP or feature importance analysis), and then select the features that have the greatest impact on the prediction based on the results. If some features suddenly become unrepresentative, or some new fields unexpectedly become more relevant to the target value, the feature set can be updated in this retraining process.

[0080] Step S207: Retrain the artificial intelligence model to generate a retrained artificial intelligence model.

[0081] In this step, after feature selection, the system will retrain the same or multiple candidate models and evaluate their performance on the new data. This process can be set to fully automatic retraining or manual retraining. When user intervention is required, the administrator can review the intermediate model error curves and metric changes to decide whether to apply the final version.

[0082] Step S208: Redeploy the retrained AI model to the application environment. After this step, return to step S201 to continuously monitor the performance of the retrained AI model.

[0083] In this step, if the retrained model outperforms the old model, the system will automatically or, after human verification, deploy it back to the original application environment to replace the degraded model. At this point, the new model can receive and predict real-time data from the application environment. If model performance degradation is detected again later, the above process can be repeated to form a continuously updated MLOps ecosystem.

[0084] like Figure 3 According to another embodiment, the method includes providing, after performing step S107, dynamic monitoring of the performance of the artificial intelligence model configured in the application environment, and re-evaluating whether the previously unselected artificial intelligence model is more suitable for the current application environment, including steps S301 to S307, as detailed below.

[0085] Step S301: Based on changes in real-time data generated from at least one data source in the application environment, monitor the performance metrics of the artificial intelligence model.

[0086] Step S302: Check if the performance indicators of the artificial intelligence model are lower than the predetermined standard. If not, proceed to step S303; if yes, proceed to step S304.

[0087] Step S303: Maintain the original AI model deployed in the application environment, and return to step S301 to continuously monitor the performance of the AI ​​model.

[0088] Step S304: Retrieve at least one previously built but not deployed AI model that failed to be selected.

[0089] Step S305: Based on the changes in the real-time data, compare the explanation results of each rejected artificial intelligence model with the model's changes.

[0090] Step S306: Select the rejected artificial intelligence model that best matches the change state of the real-time data.

[0091] Step S307: Deploy the rejected AI model to the application environment to replace the existing AI model. After this step, return to step S301 to continuously monitor the performance of the replaced AI model.

[0092] Steps S301 to S307 described above can be implemented in at least one practical application environment. Specifically, assuming a factory electricity consumption forecasting application, the system has deployed an "existing model A" on the production site to predict the electricity demand of each production line in real time. According to this embodiment, in step S301, the system continuously collects real-time data from electricity meter sensors, ERP production records, etc., and in step S302, monitors the performance indicators (such as RMSE or MAPE) of the existing model A. If the check results show that the indicators of model A are still higher than the predetermined standard (indicating low prediction error and good stability), then in step S303, the artificial intelligence model is directly maintained to continue operating, and the system returns to step S301 for continued monitoring. Conversely, if the accuracy of model A is detected to have significantly decreased (performance indicators deteriorating below the predetermined threshold), the system proceeds to step S304 to search among the multiple candidate models "previously built but not yet deployed." For example, a random forest model B and a limit-improvement model C were previously generated during the model development phase, but were not selected for formal deployment after preliminary comparison.

[0093] Therefore, in step S305, the system compares the newly collected real-time data feature distribution with the model interpretation results (e.g., SHAP value or feature importance) of each rejected model to evaluate which rejected model has a better performance advantage under the current data distribution. If the comparison finds that the random forest model B has a smaller prediction error on the recent night shift production line, then model B is selected in step S306, and in step S307, model B is deployed to the factory application environment to replace the poorly performing model A. After the replacement is completed, the entire process returns to step S301 to continuously monitor the prediction performance of the newly deployed model B on real-time data. If model B also experiences a performance decline in the future, the system can repeat the above process to automatically or semi-automatically select other more suitable models or perform a retraining mechanism. In this way, the system can flexibly switch to the artificial intelligence model that is most suitable for the current data situation without spending too much modeling time, thus maintaining overall prediction performance and operational efficiency.

[0094] The systems that execute the methods for automating the deployment of artificial intelligence models in the above embodiments include, but are not limited to, servers (such as centralized computing and data processing systems for use within enterprises or data centers), cloud platforms, edge computing devices (such as factory edge devices or smart IoT devices), high-performance computing (HPC) systems, and IoT devices.

[0095] The methods for automating the deployment of artificial intelligence models applied in the above embodiments can be executed by one or more components / components of a "system," "device," "module," or "unit." For example, they can be executed by a single processing module or by multiple modules performing different tasks, such as a data receiving module, a preprocessing module, a screening module, a modeling module, an optimization module, a performance evaluation module, or a retraining module. Furthermore, at least one of the above modules can form a system for automating the deployment of artificial intelligence models. Therefore, the methods disclosed in this embodiment are applicable regardless of whether they are in the form of a single system / device / module / unit, a system / device integrating multiple modules, or modules operating distributed across multiple components.

[0096] The above description is merely illustrative of preferred embodiments of the present invention and is not intended to limit the scope of implementation. Any simple substitutions and equivalent changes made in accordance with the scope of the patent application and the contents of the patent specification of the present invention shall fall within the scope of the patent application of the present invention.

Claims

1. A method of automating deployment of artificial intelligence models, the method comprising: include: Receive operational data relating to the operation or performance of at least one entity system, the operational data being derived from at least one data source; The received operational data is preprocessed to generate a structured dataset required for modeling; Based on at least one data feature in the structured dataset, select at least one corresponding candidate algorithm; Construct multiple artificial intelligence models, each with multiple combinations of hyperparameters, based on at least one selected candidate algorithm; Perform single-parameter optimization on these multiple artificial intelligence models; Evaluate a performance metric for each AI model and generate a corresponding model explanation result; as well as Based on the performance metrics and / or the model interpretation results, an optimized AI model is selected and deployed in an application environment corresponding to at least one data source.

2. The method of Claim 1, wherein, After deploying the AI ​​model to the application environment, the following steps are also included: Based on changes in real-time data generated from at least one data source in the application environment, the performance metric of the artificial intelligence model is continuously monitored. When the performance metric falls below a predetermined standard, the AI ​​model is automatically optimized.

3. The method of Claim 2, wherein, Optimizing this artificial intelligence model also includes the following steps: Preprocess the newly added real-time data in the application environment; Based on the newly added real-time data, if the amount of real-time data is huge, data compression without loss of quality is performed, followed by outlier cleaning and missing value processing, then the weight of the AI ​​model in judging the importance of features is updated, and features that have a significant impact on model performance are selected, so as to readjust a hyperparameter of the AI ​​model and / or a set of selected features. The AI ​​model is then retrained to generate a retrained AI model. The retrained AI model will be redeployed to the application environment.

4. The method for automatically deploying artificial intelligence models according to claim 1, characterized in that, After deploying the AI ​​model to the application environment, the following steps are also included: Based on changes in real-time data generated from at least one data source in the application environment, the performance metric of the artificial intelligence model is monitored. When the performance index falls below a predetermined standard, the following steps are performed: Retrieve at least one previously built but not deployed AI model that failed to be selected; Based on the changes in this real-time data, compare the explanation results of each rejected AI model with the data provided. Select the rejected AI model that best matches the changing state of the real-time data; Deploy the rejected AI model to the application environment to replace the existing AI model.

5. The method for automatically deploying an artificial intelligence model according to claim 1, characterized in that, The step of selecting the corresponding candidate algorithm based on the at least one data feature in the structured dataset further includes the following steps: Based on at least one data feature, at least one candidate algorithm is selected from an algorithm library; The candidate algorithm is trained using a random subset of data from the structured dataset and evaluated according to at least one performance metric. Select the at least one candidate algorithm that performs well on at least one performance metric.

6. The method for automatically deploying an artificial intelligence model according to claim 1, characterized in that, The optimization of this parameter for the multiple artificial intelligence models also includes the following steps: Adjust the multiple hyperparameters of each of the artificial intelligence models, including a learning rate, a regularization coefficient, a model structure parameter, and a batch size; Optimization is performed based on a preset parameter tuning strategy, which can be a grid search, a random search, or a heuristic optimization method.

7. The method for automatically deploying an artificial intelligence model according to claim 6, characterized in that, For each different parameter combination of the artificial intelligence model, comparisons are made based on at least one performance metric, and the parameter combination with superior performance is selected as the final parameter setting for the artificial intelligence model.

8. The method for automatically deploying an artificial intelligence model according to claim 7, characterized in that, Based on a prediction result and an internal parameter after training for each parameter combination, the model's explanation result is generated.

9. The method for automatically deploying an artificial intelligence model according to claim 1, characterized in that, The process of evaluating the performance metrics of multiple AI models and generating corresponding explanations for each model also includes the following steps: For each of the artificial intelligence models, based on a prediction result and an internal parameter, calculate the global contribution of at least one data feature to the artificial intelligence model, wherein the prediction result is an overall prediction result for the structured dataset; Based on the global contribution result, for a single data instance or a representative subset of data selected from the structured dataset, calculate a local impact value of the model prediction result for at least one data feature; Based on the calculation results of the global contribution and the local influence value, the corresponding model output value is simulated for different feature values ​​of at least one data feature, and the range of change of a model prediction result or a classification probability caused by the change of the feature value is calculated.

Images