A model quality monitoring system based on machine learning operation and maintenance

Abstract

The present invention provides a model quality monitoring system based on machine learning operation and maintenance, which belongs to the field of model quality monitoring technology. The system mainly includes a data collection module, a quality assessment module, a real-time monitoring module, an automatic adjustment module and a data verification module. Among them, the data collection module collects model parameters in the model prediction process; the quality assessment module evaluates the model performance of the model through an evaluation tool based on the model parameters, and generates a performance evaluation report according to the evaluation results; the real-time monitoring module monitors the operating status and quality indicators of the model in real time, and feeds back the operating status and quality indicators to the user; the automatic adjustment module automatically performs model optimization operations when the quality indicators of the model are lower than the preset adjustment threshold; the data verification module performs data drift detection on the distribution of model input data through a statistical test algorithm, and sends an alarm according to the detection result. The present invention can ensure the continuous and efficient operation of the model and low model operation and maintenance costs.

Description

Technical Field

[0001] The present invention relates to the technical field of model quality monitoring, and in particular to a model quality monitoring system based on machine learning operation and maintenance. Background Art

[0002] The performance of machine learning models directly affects their application effects, such as prediction accuracy and response speed. However, when models are actually deployed, their performance often declines over time, resulting in "model decay". The causes of this problem include: distribution shift between training data and actual application data, overfitting of the model, and suboptimal hyperparameters. To solve the problem of model decay, it is necessary to be able to continuously monitor and optimize the model.

[0003] Most of the model monitoring solutions currently used in the industry rely on manual work, with data scientists regularly checking logs and indicators to find problems. This approach is inefficient and difficult to meet the needs of real-time response. Some automated monitoring platforms also have the limitation of only being able to perform passive monitoring and not actively optimizing models. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a model quality monitoring system based on machine learning operation and maintenance, which is used to solve the problems that the existing model monitoring technology is difficult to automatically optimize model indicators and the model monitoring efficiency is low.

[0005] The technical solution adopted by the present invention is:

[0006] The present invention provides a model quality monitoring system based on machine learning operation and maintenance, comprising:

[0007] A data collection module is used to collect model parameters of the machine learning model during the model prediction process; the model parameters include model input data, model output module and evaluation indicators;

[0008] The quality assessment module is used to evaluate the model performance of the machine learning model based on the model parameters through the evaluation tool, and generate a corresponding performance evaluation report according to the evaluation results;

[0009] Real-time monitoring module, which is used to monitor the running status and quality indicators of the machine learning model in real time and feedback the running status and quality indicators to the user;

[0010] An automatic adjustment module, used to automatically perform model optimization operations when the quality indicator of the machine learning model is lower than a preset adjustment threshold;

[0011] The data verification module is used to perform data drift detection on the distribution of the model input data through a statistical verification algorithm and send an alarm based on the detection result.

[0012] Furthermore, the present invention also includes a model experiment module, which is used to test different versions of machine learning models selected by the user through A / B testing of single-variable evaluation indicators or full-factor testing of multi-variable combination evaluation indicators, and compare the performance of different versions of machine learning models according to the test results.

[0013] Furthermore, the present invention also includes a report generation module, which is used to generate corresponding regular evaluation reports, online monitoring reports and automatic parameter adjustment reports using interactive images based on the model performance evaluation data, data drift detection, real-time monitoring data and model optimization data of the model.

[0014] Furthermore, the data collection module collects model parameters of the machine learning model during the model prediction process, including:

[0015] Through the machine learning framework, the model parameters are collected according to the pre-configured sampling frequency when the machine learning model makes predictions, and the collected model parameters are stored in the data warehouse in a preset storage format;

[0016] The sampling frequency is specifically once per hour or per day after each prediction by the model.

[0017] Furthermore, when the quality index of the machine learning model is lower than a preset adjustment threshold, the automatic adjustment module automatically performs a model optimization operation, including:

[0018] The automatic adjustment module is connected to the real-time monitoring module through a preset data connection method to obtain quality indicators of the machine learning model, wherein the quality indicators include model accuracy, latency, event processing time, and resource utilization;

[0019] Determine whether the quality indicator is lower than the preset adjustment threshold. If the quality indicator is lower than the preset adjustment threshold, use the hyperparameter tuning algorithm to optimize the parameters of the machine learning model, automatically select the tuned new machine learning model for deployment, perform reinforcement learning on the deployed new machine learning model, and update the model parameters of the machine learning model online.

[0020] Furthermore, the hyperparameter tuning algorithm is used to optimize the parameters of the machine learning model, and the optimized new machine learning model is automatically selected for deployment, including:

[0021] When the quality index of the machine learning model is lower than the preset adjustment threshold, the abnormal type of the quality index is analyzed, and the quality index is divided into a primary index set M and a secondary index set S;

[0022] For a single main indicator Abnormal, the Bayesian optimization algorithm is used to adjust the parameters of a single main indicator. The objective function of the single main indicator parameter adjustment is:

[0023]

[0024] in, x For input data, θ is the hyperparameter vector, E[ ] is the objective function;

[0025] For multiple main indicators At the same time, if there are abnormalities, weighted multi-objective optimization is used to adjust the parameters of multiple main indicators. The objective function of adjusting the parameters of multiple main indicators is:

[0026]

[0027] in, is the weight of each indicator; mi represents the i-th main indicator;

[0028] Secondary indicators In the case of abnormality, the secondary indicators are optimized under the constraints, and the objective function of the secondary indicator parameter adjustment is:

[0029]

[0030] in, smin Secondary indicator s The lower limit of

[0031] Automatically filter N models after parameter adjustment , select the model with the best comprehensive index according to the weighted score To deploy:

[0032] .

[0033] Furthermore, the reinforcement learning of the deployed new machine learning model and online updating of model parameters of the machine learning model include:

[0034] When a new machine learning model is initially deployed, it is based on offline data. Perform heuristic search, the objective function of heuristic search is:

[0035]

[0036] in, As the evaluation index, For labels;

[0037] Define the reward function:

[0038]

[0039] in, For online data collected;

[0040] Continuously collect online data while the model is running online And train the agent network , so that the proxy network Ability to predict given hyperparameters The model is online data Expected Reward :

[0041]

[0042] Expected reward predicted by the proxy network R , use the policy gradient algorithm to optimize the parameter adjustment strategy π:

[0043]

[0044]

[0045] in, is the learning rate, To use the policy gradient algorithm from the current hyperparameters Update the new hyperparameters, is a noise function based on Gaussian distribution to prevent the strategy from falling into local optimality;

[0046] By continuously sampling , get new parameters , the new parameter Applied to the model, the model parameters are continuously updated through online evolution.

[0047] In summary, the beneficial effects of the present invention are as follows:

[0048] The present invention provides a model quality monitoring system based on machine learning operation and maintenance. By collecting data on the machine learning model and real-time tracking and monitoring of model parameters and indicators, it is convenient for developers to continuously monitor various indicators of the model and understand the operation status of the model. At the same time, the system also performs quality assessment on the model. When the quality indicators of the model decrease, it automatically performs model optimization operations and adjusts the parameters of the model, thereby improving the operating efficiency of the model and ensuring the quality of the model. In addition, the system of the present invention can also detect the distribution and changes of the model input data, issue an alarm when data drift occurs, and remind the developer to handle it, thereby improving the business processing effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.

[0050] Figure 1 This is a functional module diagram of the model quality monitoring system based on machine learning operation and maintenance of the present invention;

[0051] Figure 2 This is a schematic diagram of the operation principle of the model quality monitoring system based on machine learning operation and maintenance of the present invention;

[0052] Figure 3 It is a functional architecture diagram of the data verification module of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. If there is no conflict, the various features of the present invention and the embodiments can be combined with each other, all within the scope of protection of the present invention.

[0054] At present, in order to achieve continuous automatic monitoring and optimization of machine learning models, model quality monitoring needs to have the following functions: 1. Data collection function to track model indicators; 2. Evaluation function to determine whether the model meets the requirements; 3. Optimization function to adjust the model to improve performance; 4. Detection function for data distribution drift. Currently, there is still a lack of an overall solution that combines the above functions.

[0055] In view of the shortcomings of the above-mentioned prior art, the present invention proposes a model quality monitoring system based on the concept of machine learning operation and maintenance, which realizes the automatic and intelligent monitoring and optimization of the model. It has functional modules such as data acquisition, model evaluation, closed-loop control, model optimization and data verification, which can continuously monitor various indicators of the model, and automatically optimize when the indicators decrease. It can also detect changes in data distribution and solve the problem of online model decline. The specific implementation process of the present invention is shown in the following embodiments:

[0056] Embodiment 1:

[0057] See also Figure 1 and Figure 2 , Figure 1This is a functional module diagram of a model quality monitoring system based on machine learning operation and maintenance in Example 1 of the present invention. Figure 2 This is a schematic diagram of the operation principle of the model quality monitoring system based on machine learning operation and maintenance in Example 1 of the present invention. Figure 1 As shown, the system includes a data collection module 101, a quality assessment module 102, a real-time monitoring module 103, an automatic adjustment module 104 and a data verification module 105.

[0058] The data collection module 101 is used to collect model parameters of the machine learning model during the model prediction process; the model parameters include model input data, model output module and evaluation indicators;

[0059] The quality evaluation module 102 is used to evaluate the model performance of the machine learning model based on the model parameters through the evaluation tool, and generate a corresponding performance evaluation report according to the evaluation results;

[0060] A real-time monitoring module 103 is used to monitor the operating status and quality indicators of the machine learning model in real time, and feed back the operating status and quality indicators to the user;

[0061] An automatic adjustment module 104, configured to automatically perform a model optimization operation when a quality indicator of the machine learning model is lower than a preset adjustment threshold;

[0062] The data verification module 105 is used to perform data drift detection on the distribution of the model input data through a statistical verification algorithm and send an alarm according to the detection result.

[0063] Reference Figure 2 As shown, in the embodiment of the present invention, the operating principle of the model quality monitoring system is as follows:

[0064] S1. The data collection module 101 is integrated with the model to collect the input / output data of the model into a database.

[0065] S2. The quality assessment module 102 regularly reads data from the database, calculates assessment indicators, and generates an assessment report.

[0066] S3. The real-time monitoring module 103 monitors the operating indicators of the model in real time, such as throughput, delay, success rate, etc.

[0067] S4. If the monitoring finds that the indicator is lower than the preset threshold, the automatic adjustment module 104 is triggered.

[0068] S5. The automatic adjustment module 104 starts a new round of model training, performs parameter adjustment, hyperparameter optimization, etc., and generates a new optimized model.

[0069] S6. Redeploy the new model to the production environment. Real-time monitoring continuously tracks the metrics of the new model.

[0070] S7. The data verification module 105 verifies the input data regularly throughout the entire process to determine whether distribution deviation occurs.

[0071] S8. If a distribution change of important input features is detected, the data verification module 105 sends an alarm.

[0072] S9. The model experiment module compares multiple models through methods such as A / B testing to determine the best model.

[0073] S10. The best model is deployed online, and steps S3-S9 are repeated periodically to continuously optimize the model.

[0074] S11. All modules work together to form a closed-loop model quality management.

[0075] The embodiments of the present invention can continuously optimize the model and ensure that its performance meets the requirements through comprehensive management of data collection, quality assessment, closed-loop control, verification and iterative experiments.

[0076] When the model quality monitoring system is in use, it integrates with each model through the data collection module 101 and configures the data collection task. The quality assessment module 102, the real-time monitoring module 103 and the automatic adjustment module 104 form a closed-loop control to ensure the model performance. The model experiment module conducts rapid iterative experiments to determine the best model version. The data verification module 105 generates real-time alerts to respond to changes in data distribution in advance.

[0077] Furthermore, in an embodiment of the present invention, the system also includes a model experiment module 106, which is used to test different versions of machine learning models selected by the user through A / B testing of single-variable evaluation indicators or full-factor testing of multi-variable combination evaluation indicators, and compare the performance of different versions of machine learning models according to the test results.

[0078] Among them, the model experiment module 106 allows users to select the best version of a specific model and analyze how changes in parameters affect performance. It can track different versions of models and the relationship between model versions through model lineage. In addition, the model experiment module 106 also supports the management of a large number of model versions and maintains metadata such as configuration, parameters, performance indicators, etc. for each version. It supports multiple comparison methods: A / B testing, multivariate testing, etc., and allows the configuration of multiple evaluation indicators. An interactive version comparison interface is provided, and users can customize the comparison method.

[0079] Furthermore, in an embodiment of the present invention, the system also includes a report generation module 107, which is used to generate corresponding periodic evaluation reports, online monitoring reports and automatic parameter adjustment reports using interactive images based on the model performance evaluation data, data drift detection, real-time monitoring data and model optimization data of the model.

[0080] Among them, the report generation module 107 can generate evaluation reports, monitoring reports, tuning reports, etc. according to historical data. The reports are presented in a visual form, which is convenient for developers to browse.

[0081] Furthermore, in the embodiment of the present invention, the data collection module 101 collects model parameters of the machine learning model during the model prediction process, including:

[0082] Through the machine learning framework, the model parameters are collected according to the pre-configured sampling frequency when the machine learning model makes predictions, and the collected model parameters are stored in the data warehouse in a preset storage format;

[0083] The sampling frequency is specifically after each prediction of the model, once every hour or every day.

[0084] Specifically, the data collection module 101 supports mainstream machine learning frameworks such as TensorFlow and PyTorch. The preset storage formats are CSV, Parquet and other storage formats. All collected data is stored in a SQL or NoSQL type data warehouse. The data warehouse supports SQL query and data export. The sampling frequency, storage format, etc. of the data can be configured. An open API interface is provided to the outside world. The quality assessment module 102 and the real-time monitoring module 103 can access the data stored in the data warehouse through the API to achieve various quality management functions. The data collection module 101 collects and stores various core data for the model quality management system, provides data support for other modules, and plays the role of a data hub in the entire system.

[0085] Furthermore, in the embodiment of the present invention, when the quality index of the machine learning model is lower than the preset adjustment threshold, the automatic adjustment module 104 automatically performs a model optimization operation, including:

[0086] The automatic adjustment module 104 is connected to the real-time monitoring module 103 through a preset data connection method to obtain quality indicators of the machine learning model, wherein the quality indicators include model accuracy, latency, event processing time, and resource utilization;

[0087] Determine whether the quality indicator is lower than the preset adjustment threshold. If the quality indicator is lower than the preset adjustment threshold, use the hyperparameter tuning algorithm to optimize the parameters of the machine learning model, automatically select the tuned new machine learning model for deployment, perform reinforcement learning on the deployed new machine learning model, and update the model parameters of the machine learning model online.

[0088] Specifically, a hyperparameter tuning algorithm is used to optimize the parameters of the machine learning model, and the optimized new machine learning model is automatically selected for deployment, including:

[0089] When the quality index of the machine learning model is lower than the preset adjustment threshold, the abnormal type of the quality index is analyzed, and the quality index is divided into a primary index set M and a secondary index set S;

[0090] For a single main indicator Abnormal, the Bayesian optimization algorithm is used to adjust the parameters of a single main indicator. The objective function of the single main indicator parameter adjustment is:

[0091]

[0092] in, x For input data, θ is the hyperparameter vector, E[ ] is the objective function;

[0093] For multiple main indicators At the same time, if there are abnormalities, weighted multi-objective optimization is used to adjust the parameters of multiple main indicators. The objective function of adjusting the parameters of multiple main indicators is:

[0094]

[0095] in, is the weight of each indicator; mi represents the i-th main indicator;

[0096] Secondary indicators In the case of abnormality, the secondary indicators are optimized under the constraints, and the objective function of the secondary indicator parameter adjustment is:

[0097]

[0098] in, smin Secondary indicator s The lower limit of

[0099] Automatically filter N models after parameter adjustment , select the model with the best comprehensive index according to the weighted score To deploy:

[0100] .

[0101] Specifically, reinforcement learning is performed on the newly deployed machine learning model, and the model parameters of the machine learning model are updated online, including:

[0102] When a new machine learning model is initially deployed, it is based on offline data. Perform heuristic search, the objective function of heuristic search is:

[0103]

[0104] in, As the evaluation index, For labels;

[0105] Define the reward function:

[0106]

[0107] in, For online data collected;

[0108] Continuously collect online data while the model is running online And train the agent network , so that the proxy network Ability to predict given hyperparameters The model is online data Expected Reward :

[0109]

[0110] Expected reward predicted by the proxy network R , use the policy gradient algorithm to optimize the parameter adjustment strategy π:

[0111]

[0112]

[0113] in, is the learning rate, To use the policy gradient algorithm from the current hyperparameters Update the new hyperparameters, is a noise function based on Gaussian distribution to prevent the strategy from falling into local optimality;

[0114] By continuously sampling , get new parameters , the new parameter Applied to the model, the model parameters are continuously updated through online evolution.

[0115] In the embodiment of the present invention, the main function of the automatic adjustment module 104 is to automatically trigger the retraining or parameter adjustment of the model to improve the model performance when the monitoring system detects that the model performance index has dropped. The module is connected to the real-time monitoring system to obtain the monitoring information of the key performance indicators of the model. When the indicator is lower than the set threshold, a new model training task is automatically triggered. A variety of hyperparameter tuning algorithms are built in, such as grid search, random search, Bayesian optimization, reinforcement learning, etc. It can be optimized for indicators such as accuracy and recall rate. The newly generated model will be compared with the old model for indicators, and the model with more obvious optimization effect will be selected.

[0116] Further, in the embodiment of the present invention, if Figure 3 As shown, the data verification module 105 includes five components: a service module 201, a data format identifier 202, a summary generator 203, a benchmark module 204, and an output interpreter 205. The service module 201 is responsible for scheduling and coordination. The data format identifier 202 detects changes in the input data format. The summary generator 203 generates a summary of the input data for comparison. The benchmark module 204 saves the benchmark data. The output interpreter 205 analyzes changes in the model output and generates an alarm.

[0117] Specifically, the data format identifier 202 in the data verification module 105 adopts a statistical distribution detection algorithm to calculate the statistical characteristics of the input data and perform a KS test on the historical benchmark distribution to determine whether the format has changed significantly. The summary generator 203 uses the SHAP method to generate a summary of the input data by calculating the influence of each input feature on the model output. The benchmark module 204 stores benchmark information such as the statistical summary of historical training data and feature influence, and the benchmark is stored in a time window. The output interpreter 205 analyzes the cause of the change in model output by comparing the SHAP value of the new input data and the KL divergence with the benchmark, and quantitatively determines whether it is a data distribution drift. When distribution drift is detected, the service module 201 will send an alarm to the preset alarm system and provide a drift analysis report.

[0118] Specifically, in the embodiment of the present invention, the quality assessment module 102 is a key module in the model quality monitoring system, and its main function is to evaluate the model performance using various algorithms based on the collected model operation data. The quality assessment module 102 integrates a variety of open source tools, such as TensorFlow TFMA, PyTorch Ignite, Sklearn Eval, etc. The model calls the evaluation algorithm, calculates the extracted data, and generates various evaluation indicators. Typical evaluation indicators include: accuracy, precision, recall, F1 score, ROC curve, AUC value, etc. The final evaluation results are stored in the report database to support further analysis, such as time series analysis, comparison with benchmarks, etc. The database provides a query interface to the outside.

[0119] In an embodiment of the present invention, the model quality monitoring system realizes the automated quality management of the machine learning model. Through AI-driven monitoring, quality assessment, data verification, automatic parameter adjustment, multi-model comparison and other technical means, it ensures the continuous and efficient operation of the model in an intelligent and visual manner, with low model operation and maintenance costs and high model robustness, and assists in obtaining the optimal model configuration, so that the machine learning model can be safely and reliably applied to actual business.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model quality monitoring system based on machine learning operation and maintenance, characterized in that: include: A data collection module is used to collect model parameters of the machine learning model during the model prediction process; the model parameters include model input data, model output data and evaluation indicators; The quality assessment module is used to evaluate the model performance of the machine learning model based on the model parameters through the evaluation tool, and generate a corresponding performance evaluation report according to the evaluation results; Real-time monitoring module, which is used to monitor the running status and quality indicators of the machine learning model in real time and feedback the running status and quality indicators to the user; The automatic adjustment module is used to automatically perform model optimization operations when the quality index of the machine learning model is lower than the preset adjustment threshold, including: The automatic adjustment module is connected to the real-time monitoring module through a preset data connection method to obtain quality indicators of the machine learning model, wherein the quality indicators include model accuracy, latency, event processing time, and resource utilization; Determine whether the quality index is lower than the preset adjustment threshold. If the quality index is lower than the preset adjustment threshold, use the hyperparameter tuning algorithm to optimize the parameters of the machine learning model, automatically select the tuned new machine learning model for deployment, perform reinforcement learning on the deployed new machine learning model, and update the model parameters of the machine learning model online; The method of using a hyperparameter tuning algorithm to optimize the parameters of the machine learning model and automatically selecting the tuned new machine learning model for deployment includes: When the quality index of the machine learning model is lower than the preset adjustment threshold, the abnormal type of the quality index is analyzed, and the quality index is divided into a primary index set M and a secondary index set S; For a single main indicator Abnormal, the Bayesian optimization algorithm is used to adjust the parameters of a single main indicator. The objective function of the single main indicator parameter adjustment is: in, x For input data, θ is the hyperparameter vector, E[ ] is the objective function; For multiple main indicators At the same time, if there are abnormalities, weighted multi-objective optimization is used to adjust the parameters of multiple main indicators. The objective function of adjusting the parameters of multiple main indicators is: in, is the weight of each indicator; mi represents the i-th main indicator; Secondary indicators In the case of abnormality, the secondary indicators are optimized under the constraints, and the objective function of the secondary indicator parameter adjustment is: in, smin Secondary indicator s The lower limit of Automatically filter N models after parameter adjustment , select the model with the best comprehensive index according to the weighted score To deploy: ； A data verification module, used to detect data drift on the distribution of the model input data through a statistical verification algorithm, and send an alarm based on the detection result; The data verification module includes a service module, a data format identifier, a summary generator, a benchmark module and an output interpreter; The service module is responsible for scheduling and coordination, and sends alarms to the preset alarm system, while providing drift analysis reports; the data format identifier uses a statistical distribution detection algorithm to calculate the statistical characteristics of the input data and perform a KS test on the historical benchmark distribution to determine whether the input data format has changed significantly; the summary generator uses the SHAP method to generate a summary of the input data by calculating the influence of each input feature on the model output; the benchmark module stores benchmark information in a time window manner, and the benchmark information includes the statistical summary and feature influence of historical training data; the output interpreter analyzes the cause of the change in model output by comparing the SHAP value of the new input data and the KL divergence with the benchmark, quantitatively determines whether it is a data distribution drift, and generates an alarm when data distribution drift is detected; The report generation module is used to generate corresponding regular evaluation reports, online monitoring reports and automatic parameter adjustment reports using interactive images based on the model performance evaluation data, data drift detection, real-time monitoring data and model optimization data of the model.

2. The model quality monitoring system based on machine learning operation and maintenance according to claim 1 is characterized in that: It also includes a model experiment module, which is used to test different versions of machine learning models selected by the user through A / B testing of single-variable evaluation indicators or full-factor testing of multivariate combination evaluation indicators, and compare the performance of different versions of machine learning models based on the test results.

3. The model quality monitoring system based on machine learning operation and maintenance according to claim 1 is characterized in that: The data collection module collects model parameters of the machine learning model during the model prediction process, including: Through the machine learning framework, the model parameters are collected according to the pre-configured sampling frequency when the machine learning model makes predictions, and the collected model parameters are stored in the data warehouse in a preset storage format; The sampling frequency is specifically once per hour or per day after each prediction by the model.

4. The model quality monitoring system based on machine learning operation and maintenance according to claim 1 is characterized in that: The reinforcement learning of the deployed new machine learning model and online updating of the model parameters of the machine learning model include: When a new machine learning model is initially deployed, it is based on offline data. Perform heuristic search, the objective function of heuristic search is: in, As the evaluation index, For labels; Define the reward function: in, For online data collected; Continuously collect online data while the model is running online And train the agent network , so that the proxy network Ability to predict given hyperparameters The model is online data Expected Reward : Expected reward predicted by the proxy network R , use the policy gradient algorithm to optimize the parameter adjustment strategy π: in, is the learning rate, To use the policy gradient algorithm from the current hyperparameters Update the new hyperparameters, is a noise function based on Gaussian distribution, which prevents the strategy from falling into local optimality; by continuous sampling , get new parameters , the new parameter Applied to the model, the model parameters are continuously updated through online evolution.

Images