Machine learning modeling platform and method, electronic equipment and medium

Abstract

The invention provides a machine learning modeling platform and method, electronic equipment and a medium. The machine learning modeling platform comprises a data management module, an algorithm library module and a model training module, wherein various machine learning models are stored in an algorithm library of the algorithm library module; the data management module is used for acquiring original training data and performing preprocessing and feature engineering on the original training data to obtain training set data; the algorithm library module is used for obtaining a target model from an algorithm library in response to user operation; wherein the operation of the user comprises one of the following operations: a click operation, a drag operation and an input operation; and the model training module is used for performing model training on the target model by using the training set data based on the distributed computing framework to obtain a trained model. According to the method, the usability of the platform and the calculation efficiency of model training are improved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a machine learning modeling platform, method, electronic device and medium. Background Art

[0002] Machine learning technology is increasingly used in finance, education, healthcare, information industry and other fields, and its scope of use is becoming more and more extensive. At present, most companies hire machine learning algorithm engineers to perform modeling, which requires a lot of manpower and physical resources. In order to solve the above problems, machine learning modeling platforms have emerged. Machine learning modeling platforms integrate three functional modules: data management, visual modeling, and programmatic modeling, covering the complete process from data access to model training, evaluation, and iteration. Although existing machine learning modeling platforms have made significant progress in many aspects, they still have some shortcomings and defects. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a machine learning modeling platform, method, electronic device and medium to improve the usability of the platform and the computational efficiency of model training.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] In a first aspect, the present invention provides a machine learning modeling platform, comprising: a data management module, an algorithm library module and a model training module; wherein the algorithm library module stores a plurality of machine learning models in the algorithm library; the data management module is used to obtain original training data, and preprocess and feature engineer the original training data to obtain training set data; the algorithm library module is used to obtain a target model from the algorithm library in response to a user's operation; wherein the user's operation includes one of the following operations: a click operation, a drag operation and an input operation; the model training module is used to perform model training on the target model using the training set data based on a distributed computing framework to obtain a trained model.

[0006] Optionally, the model training module is specifically used to: perform feature selection and data encryption on the training set data to obtain training feature data; initialize model parameters of the target model; train the target model based on the training feature data, iteratively update the model parameters, and obtain a trained model.

[0007] Optionally, it also includes: a model evaluation module, which is used to calculate the performance indicators of the model during the model training process, and visualize the performance indicators, and generate model evaluation results based on the performance indicators; wherein the performance indicators include one or more of the following: loss function value, accuracy, recall rate, F1 value, AUC, weight, and gradient distribution.

[0008] Optionally, it also includes: a model optimization module, which is used to adjust the model parameters according to the model evaluation results, or to obtain a new target model according to the model evaluation results and train the new target model.

[0009] Optionally, it also includes: a model deployment module, which is used to deploy the trained model to the production environment and perform pre-launch testing and release approval on the trained model.

[0010] Optionally, it also includes: a model monitoring module, which is used to regularly obtain the operating data and performance indicators of the model that has been released and launched, and optimize and update the model that has been released and launched according to the operating data and performance indicators of the model that has been released and launched; wherein the operating data includes: input parameters and output parameters; the model monitoring module is also used to monitor the statistical indicators, distribution range, and degree of deviation of the input parameters of the model that has been released and launched, and to monitor the predicted proportion of the output parameters of the model that has been released and launched.

[0011] Optionally, it also includes: a system management module for managing trained models, including: model registration, model version management, model details registration management, workflow management, and model approval management; the system management module is also used for user and permission management, including: user permission management, user resource management, and key management.

[0012] In a second aspect, the present invention provides a machine learning modeling method, which is applied to any one of the machine learning modeling platforms provided in the first aspect, including: obtaining original training data through a data management module, and preprocessing and feature engineering the original training data to obtain training set data; obtaining a target model from an algorithm library in response to a user's operation through an algorithm library module; wherein the user's operation includes one of the following operations: a click operation, a drag operation, and an input operation; and performing model training on the target model using the training set data based on a distributed computing framework through a model training module to obtain a trained model.

[0013] In a third aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the method provided in the second aspect above.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method provided in the second aspect are executed.

[0015] The present invention brings the following beneficial effects:

[0016] The above-mentioned machine learning modeling platform, method, electronic device and medium provided by the present invention include: a data management module, an algorithm library module and a model training module; wherein the algorithm library module stores a variety of machine learning models in the algorithm library; the data management module is used to obtain the original training data, and preprocess and feature engineer the original training data to obtain the training set data; the algorithm library module is used to obtain the target model from the algorithm library in response to the user's operation; wherein the user's operation includes one of the following operations: click operation, drag operation and input operation; the model training module is used to perform model training on the target model using the training set data based on the distributed computing framework to obtain a trained model. The above-mentioned machine learning modeling platform has built-in multiple machine learning models, and the user can select the target model by clicking, dragging and inputting, and then perform model training through the distributed computing framework, thereby improving the platform's ease of use and the computational efficiency of model training, and reducing the user's learning cost.

[0017] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the preferred embodiments are specifically mentioned below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of the structure of a machine learning modeling platform provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the structure of another machine learning modeling platform provided by an embodiment of the present invention;

[0022] Figure 3 A flowchart of a machine learning modeling method provided by an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] At present, although the existing machine learning modeling platforms have made significant progress in many aspects, they still have some shortcomings and defects:

[0026] 1. Platform usability: For non-professional users, there is still a certain learning cost. At the same time, the platform is not intuitive enough in the construction and optimization of complex models, which makes it difficult for users to get started quickly.

[0027] 2. Computational efficiency issues: In the case of massive data, the modeling process often has high requirements for hardware and computing frameworks. Some platforms are not flexible enough in resource management, resulting in inefficient model training, especially when processing large-scale data sets, which may encounter performance bottlenecks.

[0028] 3. Model engineering complexity: After modelers complete model training, they usually need the assistance of the engineering team to complete the deployment and operation of the model. This not only increases the overall cost of modeling, but may also cause delays or errors in the process from model training to actual application. Some platforms still need to strengthen model deployment and integration.

[0029] 4. Algorithm and framework limitations: Some platforms have certain limitations in algorithm selection and framework support. This may cause users to be unable to find the most suitable algorithm or framework in certain specific scenarios, thus affecting the performance and accuracy of the model.

[0030] 5. Data security and privacy protection: In the process of machine learning modeling, data security and privacy protection is an issue that cannot be ignored. Some platforms may have security risks in data processing and storage, such as data leakage and illegal access, which brings potential risks to users and enterprises.

[0031] Based on this, the embodiments of the present invention provide a machine learning modeling platform, method, electronic device and medium to improve the usability of the platform and the computational efficiency of model training.

[0032] To facilitate understanding of this embodiment, a machine learning modeling platform disclosed in the embodiment of the present invention is first introduced in detail. The platform can be applied to finance, education, medical care, information industry and other fields, such as: used to build intelligent risk control models in the financial field. Figure 1The structural diagram of a machine learning modeling platform shown in the figure indicates that the platform mainly includes: a data management module 101, an algorithm library module 102 and a model training module 103; wherein, the algorithm library of the algorithm library module 102 stores a variety of machine learning models.

[0033] The data management module 101 is used to obtain original training data, and perform preprocessing and feature engineering on the original training data to obtain training set data.

[0034] In one embodiment, the machine learning modeling platform can support various data source access forms, support importing data sets from mainstream databases such as SQL Server, Oracle, Inceptor, JDBC, HBase, Inceptor / Hive, ArgoDB, db2jcc4, HBase, MySQL, etc., and support creating new drivers. It also supports debugging and compatibility with various data sources such as the organization's self-developed data storage, as well as importing data sets from local files, file management and sharing, or importing from API interfaces.

[0035] After obtaining the original training data, the data management module 101 can preprocess and feature engineer the original training data. First, the collected data is preprocessed, mainly denoising, processing missing values, outliers or duplicate values, and type conversion operations to ensure the quality and accuracy of the data. For example: in calculating the transaction amount variation and slope, remove the transaction amount at the specific time of Double Eleven and Double Twelve; in the data preprocessing process, it is also necessary to convert character type data into numerical type according to the meaning of each field. Among them: binary data: taking the sex field as an example, its value is female or male, 0 can be used to represent female, and 1 can be used to represent male; multi-value data: taking the cp field as an example, this parameter represents chest pain, and the pain can be mapped from light to heavy to values ​​0 to 3; then feature engineering is performed, including: data standardization or normalization, etc.

[0036] In the specific implementation, the features and labels of the pre-processed original training data are managed, including: feature extraction (extracting meaningful features from the original data), feature expansion (adding new features when there are fewer features) to increase or decrease the number of features, thereby improving the prediction ability of the model. Specifically, feature selection, construction and conversion can be performed according to the business problems and data characteristics related to the model to be built. Taking the construction of an intelligent risk control model as an example, user features can be extracted from the user's dynamic behavior data and the user's static asset data to form user portrait labels, totaling hundreds of features, such as payment and purchase data. In general, the more effective features and label data are: account balances in the past 1 / 3 / 6 / 12 months, the maximum transaction interval in the past 1 / 3 / 6 / 12 months, the coefficient of variation of the transaction amount in the past 1 / 3 / 6 / 12 months, the number of days with transactions in the past 1 / 3 / 6 / 12 months, the sum of the transaction amount in the past 1 / 3 / 6 / 12 months, and the slope of the transaction amount in the past 1 / 3 / 6 / 12 months.

[0037] Furthermore, feature engineering is performed. Feature engineering mainly includes the derivation and scale change of features, etc., mainly including: data standardization and normalization. According to the characteristics of the data, an appropriate data normalization method is selected, and the normalization component is used to convert the numerical range of each feature to 0-1, thereby removing the influence of the dimension on the result. The formula is result = (val-min) / (max-min), where result represents the normalization result, val represents the current value of the feature, min represents the minimum value of the feature, and max represents the maximum value of the feature. For example: when the data is normally distributed, StandardScaler can be used, and when the data is non-Gaussian distributed, MinMaxScaler can be used to make different features comparable.

[0038] Specifically, feature engineering includes the following steps:

[0039] Step 1: Convert character features into numerical features through the one-hot encoding component.

[0040] Step 2: Use the normalization component to limit all data to between 0 and 1 to remove the dimension effect.

[0041] Step 3: Use the SQL script to mark the target columns that are anomaly as 1 and the target columns that are normal as 0.

[0042] In the embodiment of the present invention, interactive data preprocessing is supported. The data management module 101 has a large number of general feature engineering operators built in. Users can select the required feature engineering operators to process data and construct features through a visual interface. At the same time, automatic feature engineering operators are also provided to support automatic construction of high-order combined features based on original features and targets.

[0043] In addition, in this embodiment, a preliminary exploratory analysis can be performed on the data to analyze the distribution, characteristics, missing values, etc. of the modeling features, verify the effectiveness of feature derivation and construction, and prepare for feature engineering and data standardization or normalization.

[0044] The algorithm library module 102 is used to obtain a target model from the algorithm library in response to a user's operation; wherein the user's operation includes one of the following operations: a click operation, a drag operation, and an input operation.

[0045] In one embodiment, the algorithm library of the algorithm library module 102 has more than 100 distributed operators built in, including commonly used machine learning and deep learning models. The machine learning models include at least: linear regression, logistic regression, linear discriminant analysis, naive Bayes, KNN, random forest, XGBoost, LightGBM, Catboost; the deep learning models include at least: TensorFlow, PyTorch, Scikit-learn, MLflow and other training framework models.

[0046] In the embodiment of the present invention, the machine learning modeling platform provides a unified visualization, drag-and-drop data analysis and mining web and client interface through the underlying docking with Spark, Tensorflow, Caffe, MXnet and other frameworks, provides tool support for model training, optimization, and rapid iteration, and greatly shortens the cycle of model verification and iteration. It supports user visual modeling and programmatic modeling, and supports three automatic modeling modes: click-type, drag-and-drop, and coding. Through simple drag-and-drop operations and a small amount of parameter configuration, users can select the required model according to actual needs, customize the addition of operators (i.e., target models), and combine multiple models. Among them, the custom configuration includes complex reasoning logic of multiple models, such as data pre-processing and post-processing logic, routing logic, combination logic and other processing methods. The platform can cover the work needs of various modeling personnel, and has built-in automatic modeling templates for different scenarios, which facilitates rapid replication and accurate application of similar scenarios. In addition, the platform also provides functions such as model export, model management, and model sharing, and can provide a variety of modeling scenario templates and one-stop interface operations to guide users to quickly create business scenario-related models.

[0047] Furthermore, the algorithm library module 102 also includes: custom algorithm components, such as SQL scripts, Python scripts, PyAlink scripts, etc., which can meet the needs of customized algorithms. The platform supports the import and management of PMML and pkl model files, unified management of multi-source heterogeneous algorithm frameworks, model files and model services, and realizes the deployment of different platforms and offline models, increasing the compatibility of models.

[0048] The model training module 103 is used to perform model training on the target model using the training set data based on the distributed computing framework to obtain a trained model.

[0049] In one embodiment, the model training module 103 is specifically used to:

[0050] First, feature selection and data encryption are performed on the training set data to obtain training feature data.

[0051] In specific implementation, the model training module 103 supports the Stepwise feature selection function. Stepwise is a fusion of forward selection and backward selection, that is, each time a new variable is added to the model by forward feature selection, a backward selection is required for the variables that have entered the model to remove variables whose significance does not meet the requirements. Since multiple objective functions and multiple feature transformation methods are supported at the same time, the Stepwise feature selection process supports the following multiple selection criteria:

[0052] 1) Marginal Contribution: Applicable to all objective functions and feature engineering methods.

[0053] Model A does not contain variable X, and model B contains all variables of A and variable X. The difference in the corresponding objective functions when the two training models finally converge is the marginal contribution of variable X among all variables in model B. In the scenario where feature engineering is Dummy transformation, the marginal contribution of the original variable X is defined as the difference in the objective functions of all Dummy variables of the two models that include and do not include the variable. Therefore, feature selection using marginal contribution supports all feature engineering methods.

[0054] The advantage of this method is that it is flexible and not limited to a certain model. It directly selects variables that make the objective function better into the model. The marginal contribution is different from statistical significance. Statistical significance usually chooses 0.05 as the threshold, while the marginal contribution does not have an absolute conceptual threshold for new users, and it can usually be set to 10E-5.

[0055] 2) Score Test: Only supports WOE transformation or logistic regression without feature engineering.

[0056] In the forward selection process, a model with only an intercept term is first trained. In each subsequent iteration, the Score Chi-Square statistic is calculated for the variables that have not entered the model, and then the variable with the largest Score Chi-Square statistic is selected into the model. At the same time, the significance P Value corresponding to the statistic is calculated based on the Chi-Square distribution. If the P Value of the variable with the largest Score Chi-Square statistic is greater than the maximum significance threshold (slentry) specified by the user for entering the model, the variable will not be included in the model and the selection process will stop.

[0057] After completing a round of forward selection, a round of backward selection will be performed on the variables that have been selected to enter the model. During the backward selection process, the corresponding Wald Chi-Square statistics (Wald Chi-Square) and the corresponding significance P Value are calculated for the variables that have entered the model. If the PValue is greater than the maximum significance threshold (slstay) specified by the user to remove the model, the variable is removed from the model and the next round of iterative selection is continued.

[0058] 3) F Test: Only supports WOE transformation or linear regression selection without feature engineering.

[0059] In the forward selection process, a variable with only an intercept term is first trained. In each subsequent iteration, the F Value of the variable that does not enter the model is calculated. The calculation of F Value is similar to the calculation of marginal contribution. Two models need to be trained to calculate the F Value of a variable. F Value conforms to the F distribution, and its corresponding significance P Value can be obtained based on the probability density function of its F distribution. If the P Value is greater than the maximum significance threshold (slentry) specified by the user for entering the model, the variable will not be included in the model and the selection process will stop.

[0060] The backward selection process also uses F Value to calculate significance, and the process is similar to the scoring test.

[0061] In an embodiment of the present invention, in order to avoid data leakage, the training feature data can be encrypted by using a symmetric encryption algorithm, an asymmetric encryption algorithm, a hash function, a hybrid encryption algorithm, or the like, or the data can be desensitized or anonymized, and model training can be performed based on the encrypted training feature data to ensure data security.

[0062] Then, the model parameters of the target model are initialized.

[0063] In specific implementation, the model training module 103 configures parameters for the selected target model. The model parameters of the machine learning and deep learning algorithms generally include: learning rate, regularization term, batch size, number of iterations, etc.

[0064] Finally, the target model is trained based on the training feature data, and the model parameters are iteratively updated to obtain a trained model.

[0065] In the specific implementation, the model is trained using the training feature data. In each iteration, the model will iteratively update the model parameters through optimization algorithms (such as gradient descent, Newton's method, etc.) according to the features and labels of the training data to improve its prediction performance. The model performance can also be evaluated by minimizing the loss function (i.e., the difference between the predicted value and the actual value), and the model parameters or structure can be adjusted according to the evaluation results to perform model tuning to improve its generalization ability and performance.

[0066] During the model training process, the platform will calculate the model's loss function, evaluate the model's performance, and adjust the model parameters or select other models for trial based on the evaluation results. The methods involved include optimization algorithms such as gradient descent and back propagation, as well as regularization, dropout and other strategies to prevent overfitting. Model training process monitoring mainly includes the following processes:

[0067] 1) Training error and validation error:

[0068] By monitoring the changing trends of the training error and the validation error, the performance of the model on the training set and the validation set can be understood. A decrease in the training error usually means that the model is learning, while a stable or decreasing validation error indicates that the model has good generalization ability. Based on this, in the embodiments of the present invention, the model parameters can be adjusted according to the changing trends of the training error and the validation error.

[0069] 2) Loss function value:

[0070] The loss function value reflects the difference between the model's predicted value and the actual value. Monitoring the change in the loss function value can help determine whether the model is optimizing in the right direction. If the loss function value continues to decrease during training, it means that the model is gradually optimizing; if the loss function value fluctuates greatly or stagnates, you may need to adjust parameters such as the learning rate and optimization algorithm.

[0071] 3) Learning rate adjustment:

[0072] The learning rate is one of the key parameters in the model training process, which determines the step size of parameter updates. Monitoring the changes in the learning rate can help determine whether the model is updating parameters at an appropriate speed. If the learning rate is too high, the model may oscillate and fail to converge during training; if the learning rate is too low, the training process may be too slow or fall into a local optimum. Therefore, the learning rate needs to be adjusted according to the actual situation.

[0073] 4) Weight and gradient distribution:

[0074] Monitoring the weight and gradient distribution of the model can help you understand the training status of the model. Changes in weight distribution can reflect the learning process of the model, while the distribution of gradients can reveal the optimization direction of the model. If there are problems such as abnormal weight distribution or gradient disappearance / explosion, you need to adjust the model structure or optimize the algorithm in time.

[0075] 5) Hardware indicator monitoring:

[0076] During model training, you also need to pay attention to changes in hardware indicators, such as CPU / GPU usage, memory usage, etc. These indicators can reflect the model's computing efficiency and resource consumption. If hardware resource usage is too high or too low, you may need to adjust the model's batch size, parallel computing strategy and other parameters to improve training efficiency.

[0077] The above-mentioned machine learning modeling platform provided by the present invention has built-in multiple machine learning models. Users can select the target model by clicking, dragging and inputting, and then perform model training through a distributed computing framework, thereby improving the platform's usability and the computational efficiency of model training and reducing the user's learning cost.

[0078] See also Figure 2 As shown, the above platform also includes: a model evaluation module 104, which is used to calculate the performance indicators of the model during the model training process, and visualize the performance indicators, and generate model evaluation results based on the performance indicators; wherein the performance indicators include one or more of the following: loss function value, accuracy, recall rate, F1 value, AUC, weight, gradient distribution.

[0079] In specific implementation, the model evaluation module 104 can monitor the training process during the model training process and calculate the performance indicators of the model, such as: loss function value, accuracy, recall rate, F1 value, AUC, weight, gradient distribution; the user can select the appropriate indicator according to the specific task. If the model performance does not meet the requirements, return to adjust the model structure or parameters, retrain and evaluate. For example, calculate the loss function value during the model training process to reflect the degree to which the model is close to the target solution; calculate the accuracy of the model in predicting classification problems; and evaluate the quality of the model offline through the AUC indicator.

[0080] The training process of the model can be visualized through tools such as TensorBoard, including visualization of loss function values, accuracy, weight histograms, etc. Its statistical characteristics and distribution can be viewed in TensorBoard. TensorBoard's visualization display charts can intuitively understand the training status and performance changes of the model. When debugging and optimizing the model, it can effectively monitor and adjust the training process to ensure that the model training proceeds smoothly and achieves the expected performance goals.

[0081] In another embodiment, the model evaluation module 104 can also adopt a variety of evaluation methods, such as: hold-out method, cross-validation method and bootstrap method, etc., use the validation set to evaluate the trained model, check its performance on unseen data, and use the "test error" of the test set to approximate the generalization error of the model. The evaluation results of these evaluation methods are used to compare the performance of different models, so as to select the optimal model.

[0082] The model evaluation module 104 can organize the evaluation results into an evaluation report, including the analysis results of various indicators, and display the evaluation indicator data and the changes relative to the original values ​​in the evaluation report to evaluate the performance of the model. The evaluation report can display the key information and results of the modeling process to the user in a visual way, which can help developers understand and improve the model and ensure that the model is effective, reliable and meets expectations in practical applications.

[0083] Furthermore, during the model training process, the model evaluation module 104 can also monitor changes in hardware indicators, such as CPU / GPU usage, memory usage, etc. These indicators can reflect the computational efficiency and resource consumption of the model. If the hardware resource usage is too high or too low, it may be necessary to adjust the model's batch size, parallel computing strategy and other parameters to improve training efficiency.

[0084] See also Figure 2 As shown, the above platform also includes: a model optimization module 105, which is used to adjust the model parameters according to the model evaluation results, or to obtain a new target model according to the model evaluation results and train the new target model.

[0085] In specific implementation, the model optimization module 105 can provide model optimization suggestions based on the evaluation results, including parameter adjustment, algorithm replacement and other operations, to help users improve model performance and shorten the model verification and iteration cycle.

[0086] Specifically, the model parameters can be adjusted through automated search algorithms to find the best hyperparameter combination. Such as grid search, random search or Bayesian optimization, etc., to find the optimal solution within the specified parameter range. You can also choose a suitable optimization method based on the nature of the loss function (evaluation indicator), such as gradient descent, stochastic gradient descent, etc., to continuously adjust the model parameters and find the best performance. Commonly used optimization algorithms include:

[0087] L-BFGS: A first-order optimization algorithm that supports large-scale feature data sets. This algorithm is an unconstrained optimization algorithm that automatically ignores constraints.

[0088] Newton's Method: Newton's method is a classic second-order algorithm with fast convergence speed and high accuracy. However, since it needs to calculate the second-order Hessian Matrix, it is not suitable for larger feature scales. This algorithm is an unconstrained optimization algorithm and automatically ignores constraints.

[0089] Barrier Method: A second-order optimization algorithm that is completely equivalent to Newton's method when there are no constraints. The computational performance and accuracy of this algorithm are not much different from SQP, so SQP is usually recommended.

[0090] SQP (Sequential Quadratic Programming): A second-order optimization algorithm that is completely equivalent to Newton's method when there are no constraints. The computational performance and accuracy of this algorithm are not much different from those of BarrierMethod, so SQP is usually recommended.

[0091] In another embodiment, if after model evaluation and optimization, the prediction accuracy, generalization ability, stability, etc. of the selected target model do not meet the requirements, the algorithm can be replaced, that is, a new target model is selected, and the model training and performance evaluation of the target model are re-performed, and the new target model is optimized according to the evaluation results.

[0092] See also Figure 2 As shown, the above platform also includes: a model deployment module 106, which is used to deploy the trained model to the production environment, and perform pre-online testing and release approval on the trained model.

[0093] In specific implementation, the model deployment module 106 can deploy the optimized model to the production environment, provide model integration, testing and monitoring functions, ensure the stability and accuracy of the model in practical applications, and automatically trigger the approval process at key nodes in the model deployment process to avoid waste of service resources. The model deployment module supports automatic deployment and release of models, and can easily deploy models anytime and anywhere; it supports multi-version traffic distribution strategies. Through grayscale release and independent release strategies, users can choose service versions as needed and configure the corresponding traffic ratio, thereby ensuring the smooth upgrade of services, and having grayscale release and A / B testing capabilities.

[0094] The model deployment module 106 can deploy the trained model to the actual environment, such as a web server, mobile application or embedded device. The model deployment module supports one-click model deployment, the hardware, operating system and network conditions of the production environment, and adapts to various model formats and deployment methods. The model deployment module can provide deployment tools and frameworks. Users can choose suitable machine learning frameworks and deployment tools based on the needs and resources of the model. These tools and frameworks should support model loading, reasoning and updating, so that users can integrate models into business systems.

[0095] After the model is deployed to the production environment and before it goes online, it is necessary to test the deployed model in the production environment to verify its performance and stability. The model deployment module 106 supports single data testing and outputs prediction results based on the data input by the user, thereby performing real-time inspection of the deployed model and performing service testing before going online.

[0096] After the test, you can enter the approval process to approve the model online process. Specifically, when the cluster opens the approval function, ordinary users need to submit an approval application to the administrator when triggering key node operations such as "service online / offline" and "editing online services". The operation content can only be executed after the administrator's review and approval. After approval, the model can be put online for actual application.

[0097] See also Figure 2 As shown, the above platform also includes: a model monitoring module 107, which is used to regularly obtain the operating data and performance indicators of the model that has been released and launched, and optimize and update the model that has been released and launched according to the operating data and performance indicators of the model that has been released and launched; wherein the operating data includes: input parameters and output parameters.

[0098] In specific implementation, the model monitoring module 107 can monitor the performance of the model in a real production environment and collect new data to optimize and update the model. For the model service that has been released online, the model monitoring module can regularly collect the operation data and performance indicators of the model for analysis and evaluation to ensure its performance and stability. Specifically, the model monitoring module can connect to the upstream business system data through the API interface to achieve real-time model prediction and optimize the iterative model.

[0099] The model monitoring module 107 can help users control the frequency of model iteration and complete the scheduled iteration of the model by providing cycle management for task flows such as ETL processing, model training, and model listing. At the same time, combined with the model launch system of the container, it helps users to more easily achieve rolling release and horizontal expansion, and supports automatic model iteration to continuously maintain the model application effect. It also supports one-stop model deployment and operation and maintenance, and flexibly expands model prediction services through containerized architecture, helping users to quickly implement model deployment, application, monitoring, operation and iteration.

[0100] Data drift is one of the important reasons why model accuracy decreases over time, that is, due to changes in the distribution of input data, the model prediction performance decreases. Therefore, monitoring the data drift of the model service helps to detect the model prediction performance, and timely warn and prompt, and perform subsequent model iterations. Based on this, the model monitoring module 107 is also used to monitor the statistical indicators, distribution range, and degree of drift of the input parameters of the released online model, as well as the predicted ratio of the output parameters of the released online model.

[0101] In the specific implementation, (1) the statistical indicators of the input parameters are monitored: according to the feature statistical mean, standard deviation, maximum value, and minimum value; (2) the distribution range of the input parameters is monitored: according to the distribution comparison of the feature statistical data; (3) the deviation degree of the input parameters is monitored: according to the feature statistical data deviation indicator, including PSI and JS divergence; (4) the predicted proportion of the output parameters is monitored: the proportion of each type of output is counted (only for classification models).

[0102] For data deviation indicators such as PSI and JS divergence, and service resource indicators such as memory occupancy and CPU occupancy, early warning rules and notification prompts can be set in the embodiment of the present invention. If it is found that the performance indicators of the model have declined or other problems have occurred, a warning will be issued in time and relevant personnel will be notified to conduct investigation and adjustment.

[0103] In addition, the model monitoring module 107 also supports scenario-based monitoring, with more than 40 built-in monitoring reports, covering the monitoring needs of scenario models such as retail internal rating, non-zero internal rating, and online lending, and building a fully automatic model monitoring and early warning system to continuously track the operation of the model, predict risks in advance, and issue early warnings in time so that reasonable measures can be taken. Scenario-based monitoring reports also provide customers with ideas and tools for interpreting and analyzing business data, helping customers quantify business results.

[0104] See also Figure 2 As shown, the above platform also includes: a system management module 108, which is used to manage the trained model, including: model registration, model version management, model details registration management, workflow management, and model approval management.

[0105] In specific implementation, the system management module 108 can manage the trained model, including: model registration, model version management, model details registration management, workflow management, and model approval management. The system management module 108 supports multi-version management of model images, and one image can contain multiple sub-versions. Users can view the image version details as needed, or publish the required image as a service. During the entire modeling process, the system management module 108 can also record detailed log information, including: the execution status of each link such as data loading, model training, evaluation, and deployment.

[0106] Furthermore, the system management module 108 is also used for user and authority management, including: user authority management, user resource management, and key management.

[0107] In specific implementation, the system management module 108 can configure the user resource upper limit and perform key management, support the configuration of the user group resource upper limit, and support the viewing of resource pool usage.

[0108] From a security perspective, the system management module 108 can also provide a variety of user authority controls to meet the restrictions on cluster environment, resources and system functions of different teams and different user role levels, provide privacy and security protection, and set user permissions for file management, including separate settings for cluster distributed files and network file systems.

[0109] The machine learning modeling platform of the embodiment of the present invention provides full life cycle management of data access, preprocessing, feature engineering, model training, model evaluation and model iteration, allowing users to complete machine learning modeling without writing code, effectively lowering the threshold for use. Compared with existing modeling platforms, it has the following advantages:

[0110] (1) Platform usability and user experience: The machine learning platform reduces the user's learning cost by providing a more intuitive and user-friendly operating interface and interactive method. It also uses drag-and-drop interactive modeling and automated modeling functions to enable non-professional users to quickly build and optimize models.

[0111] (2) Computational efficiency and resource optimization: The machine learning platform integrates advanced distributed computing frameworks and algorithm optimization technologies, significantly improving the computational efficiency of model training. Through intelligent scheduling and resource management, it can rationally utilize computing resources, reduce resource waste, and lower modeling costs.

[0112] (3) Support for large model technology: In response to the rapid development of large model technology, the machine learning platform provides a more flexible and scalable architecture to support the access, fine-tuning, deployment and application of large models. The platform also provides a wealth of open source large models and applications to facilitate users to quickly start large model technology research and development.

[0113] (4) Model engineering and application integration: The machine learning platform provides one-stop model development, training, and deployment services, simplifying the model engineering process. The platform supports local deployment and container deployment, facilitating users to conduct secondary development and application integration, and accelerating the transformation process from model training to application.

[0114] (5) Data security and privacy protection: The machine learning platform has strengthened its support for data security and privacy protection, ensuring the security of user data through technical means such as data encryption and access control. It also provides data desensitization and anonymization processing functions to further protect user privacy.

[0115] For the machine learning modeling platform provided in the above embodiments, the present invention provides a machine learning modeling method, see Figure 3 The flowchart of a machine learning modeling method shown in FIG. 1 illustrates that the method mainly includes the following steps S301 to S302:

[0116] Step S301: obtaining original training data through a data management module, and performing preprocessing and feature engineering on the original training data to obtain training set data;

[0117] Step S302: acquiring a target model from an algorithm library in response to a user's operation through an algorithm library module; wherein the user's operation includes one of the following operations: a click operation, a drag operation, and an input operation;

[0118] Step S303: Based on the distributed computing framework, the model training module uses the training set data to perform model training on the target model to obtain a trained model.

[0119] The above-mentioned machine learning modeling method provided by the present invention is based on a variety of machine learning models built into the platform. Users can select the target model by clicking, dragging and inputting, and then perform model training through a distributed computing framework, thereby improving the platform's usability and the computational efficiency of model training and reducing the user's learning cost.

[0120] It should be noted that the implementation principle and technical effects of the method provided in the embodiment of the present invention are the same as those of the aforementioned modeling platform embodiment. For the sake of brief description, for matters not mentioned in the method embodiment, reference may be made to the corresponding contents in the aforementioned modeling platform embodiment.

[0121] An embodiment of the present invention further provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above implementation methods.

[0122] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 40, a memory 41, a bus 42 and a communication interface 43, wherein the processor 40, the communication interface 43 and the memory 41 are connected via the bus 42; the processor 40 is used to execute an executable module stored in the memory 41, such as a computer program.

[0123] The memory 41 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 43 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0124] The bus 42 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0125] Among them, the memory 41 is used to store programs, and the processor 40 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 40 or implemented by the processor 40.

[0126] The processor 40 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 40. The above processor 40 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to execute, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 41, and the processor 40 reads the information in the memory 41 and completes the steps of the above method in combination with its hardware.

[0127] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0128] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0129] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A machine learning modeling platform, characterized in that: include: A data management module, an algorithm library module and a model training module; wherein the algorithm library of the algorithm library module stores a variety of machine learning models; The data management module is used to obtain original training data, and perform preprocessing and feature engineering on the original training data to obtain training set data; The algorithm library module is used to obtain the target model from the algorithm library in response to the user's operation; wherein the user's operation includes one of the following operations: a click operation, a drag operation, and an input operation; The model training module is used to perform model training on the target model using the training set data based on a distributed computing framework to obtain a trained model.

2. The machine learning modeling platform according to claim 1, characterized in that: The model training module is specifically used for: Performing feature selection and data encryption on the training set data to obtain training feature data; Initializing model parameters of the target model; The target model is trained based on the training feature data, and the model parameters are iteratively updated to obtain a trained model.

3. The machine learning modeling platform according to claim 1, characterized in that: Also includes: The model evaluation module is used to calculate the performance indicators of the model during the model training process, visualize the performance indicators, and generate model evaluation results based on the performance indicators; wherein the performance indicators include one or more of the following: loss function value, accuracy, recall rate, F1 value, AUC, weight, and gradient distribution.

4. The machine learning modeling platform according to claim 3, characterized in that: Also includes: The model optimization module is used to adjust the model parameters according to the model evaluation results, or to obtain a new target model according to the model evaluation results and train the new target model.

5. The machine learning modeling platform according to claim 1, characterized in that: Also includes: The model deployment module is used to deploy the trained model to the production environment and perform pre-launch testing and release approval on the trained model.

6. The machine learning modeling platform according to claim 5, characterized in that: Also includes: A model monitoring module is used to regularly obtain the operating data and performance indicators of the model that has been released and launched, and optimize and update the model that has been released and launched according to the operating data and performance indicators of the model that has been released and launched; wherein the operating data includes: input parameters and output parameters; The model monitoring module is also used to monitor the statistical indicators, distribution range, and deviation degree of the input parameters of the released online model, and to monitor the predicted proportion of the output parameters of the released online model.

7. The machine learning modeling platform according to claim 1, characterized in that: Also includes: The system management module is used to manage the trained models, including: model registration, model version management, model details registration management, workflow management, and model approval management; The system management module is also used for user and authority management, including: user authority management, user resource management, and key management.

8. A machine learning modeling method, characterized in that: The machine learning modeling platform applied to any one of claims 1 to 7, comprising: Obtaining original training data through a data management module, and performing preprocessing and feature engineering on the original training data to obtain training set data; Responding to the user's operation through the algorithm library module, obtaining the target model from the algorithm library; wherein the user's operation includes one of the following operations: a click operation, a drag operation, and an input operation; The model training module is based on a distributed computing framework and uses the training set data to perform model training on the target model to obtain a trained model.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the method according to claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 8 are performed.

Images