Methods and systems for integrating databases with online machine learning inference services

Abstract

This invention discloses a method and system for integrating a database with an online machine learning inference service, belonging to the field of artificial intelligence technology. The technical problem it addresses is how to combine machine learning models with a database and lower the barrier to entry for users of machine learning models. The method includes: constructing built-in SQL functions in the database and defining the definition method of these SQL functions; providing model management and online prediction services through these SQL functions; constructing built-in model permission levels and management functions in the database and defining the calling methods for these functions; constructing model details tables and permission details tables within the database; and constructing an online inference service, the process of which includes a model deployment process and an online prediction process.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and system for integrating databases with online machine learning inference services. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) and communication technologies, massive amounts of heterogeneous data are constantly being generated and imported into database systems for management. Users hope to leverage machine learning and artificial intelligence (AI) technologies to perform in-depth predictive analysis on this data to maximize its value and assist in user decision-making. However, due to the vastly different reasoning methods of various models, ordinary users still face a significant barrier to using AI technology. Therefore, people hope to use AI through simple operations. Furthermore, in many scenarios where AI technology is already widely used, there is a desire to improve operational and maintenance efficiency by simplifying its use, and to reduce the development and usage costs of AI technology by adopting universal deployment methods.

[0003] KServe is currently a widely used machine learning deployment framework. This framework is quite comprehensive, supports mainstream machine learning models, and also offers users considerable flexibility. However, this high flexibility means users must pay attention to many technical details when deploying models, which presents a significant challenge and inevitably leads to a dispersion of user focus, preventing them from concentrating on the business attributes of the model. This is especially true in large-scale data scenarios, where there is a strong desire to further lower the barriers to entry for AI-related algorithms.

[0004] How to combine machine learning models with databases and lower the barrier for users to use machine learning models is a technical problem that needs to be solved. Summary of the Invention

[0005] The technical objective of this invention is to address the above-mentioned shortcomings by providing a method and system for integrating databases with online machine learning inference services, thereby solving the technical problem of how to combine machine learning models with databases and lowering the barrier for users to use machine learning models.

[0006] In a first aspect, the present invention provides a method for integrating a database with an online inference service for machine learning, applicable to a client, a database, and an online inference service, comprising the following steps:

[0007] Model and Online Prediction Management: Construct built-in SQL functions in the database and define the definition method of SQL functions. Provide model management services and online prediction services through SQL functions. The model management service supports model import, model list viewing, model details viewing, and model deletion. The online prediction service is used to verify the permissions of online prediction requests initiated by users through the client and push the online prediction requests that have passed the permission verification to the online inference service.

[0008] Model permission management: Construct database-built-in model permission levels and management functions, and define the calling methods of model permission levels and management functions. Provide model permission level management services through model permission levels and management functions. The model permission level management service supports setting model permission levels, setting user roles and permissions, and managing model permissions. Among them, model permission management supports granting, revoking and viewing permissions for users to execute machine learning models.

[0009] Database table design: Build a model details table and a permission details table within the database. For machine learning models imported into the database, the model details table is used to store the model and model metadata, and the permission details table is used to store the model permission level of each machine learning model, the role of each user, and the permissions for the machine learning model.

[0010] Online Inference Service Management: Building an online inference service involves two processes: model deployment and online prediction. The model deployment process includes storing and managing model-related files for the machine learning model specified in the online prediction request, as well as creating an inference service instance. The inference service instance is used to execute machine learning model deployment. The online prediction process includes detecting the status of the inference service instance and the prediction processing logic. The prediction processing logic is used to make predictions based on the machine learning model and return the prediction results to the database.

[0011] Preferably, the database and the online inference service communicate via the gRPC mechanism.

[0012] Preferably, the SQL functions in the database used to implement management services and online prediction services include the model list viewing function show_models, the model import function import_model, the model details viewing function show_model, the online prediction function predict, and the model deletion function remove_model;

[0013] Model permission levels and management functions include the model permission granting function `grant_privilege_on_model`, the model permission cancellation function `revoke_privilege_on_model`, and the model permission viewing function `has_model_privilege`.

[0014] Model permission levels include ALL, SELECT, CREATE, DELETE, and PREDICT. The built-in functions corresponding to the SELECT permission are Show_models and Show_model. Show_models is used to view the list of machine learning models, and Show_model is used to view the details of machine learning models. The built-in function corresponding to the CREATE permission is Import_model, which is used to import machine learning models. The built-in function corresponding to the DELETE permission is Remove_model, which is used to delete machine learning models. The built-in function corresponding to the Predict permission is PREDICT, which is used for online prediction.

[0015] User roles include system administrator and owner. The system administrator role has ALL permissions for all machine learning models by default, and the owner role has ALL permissions for importing machine learning models by default. System administrators and owners can grant machine learning model permissions to a user or role, but users cannot transfer their permissions to other users. System administrators and model creators have permission to call the model permission granting function and the model permission revocation function, while system administrators, model creators, and model users have permission to call the model permission viewing function.

[0016] Preferably, the online prediction service is used to perform the following permission verification for online prediction requests initiated by users through the client:

[0017] Verify if the current user is a system administrator. If so, pass the permission verification directly; otherwise, proceed to the next step.

[0018] Query the current user's role and the permission details table to verify whether the user's role and permissions match the current operation permission requirements. If they match, the permission verification passes; otherwise, the permission verification fails and the service is denied.

[0019] As a preferred option, an online inference service is built on KServe, and the online predictive analytics workload runs as a cloud service on the Kubernetes platform.

[0020] In the model deployment process, the model-related files of the machine learning model specified in the line prediction request are written to the local PVC shared file system of Kubernetes for unified management. For each stored machine learning model, a new folder is created and the folder name is used as a unique identifier. The folder name format is model name-model version. A single machine learning model folder has one or two subdirectories. The model subdirectory is required and is used to store model entities and static files. The data-preparation directory is optional and is used to store user preprocessing logic, processing logic and related dependencies.

[0021] Each inference service consists of a Predictor component and a Transformer component. The data preprocessing stage and the prediction result postprocessing stage are deployed in the Transformer component, and the model is deployed in the Predictor component. Each component is a container in Kubernetes. When it starts, it needs to load a base image and a directory mounted in the Kubernetes local PVC shared file system. When the Predictor component starts, it needs to mount the model directory under the local folder of the model to the fixed directory of the container. When the Transformer component starts, it needs to mount the data-preparation directory under the local folder of the machine learning model to the fixed directory of the container.

[0022] In the online prediction process, the status of the inference service instance is checked. If the status of the inference service instance is ready, it means that the service is available and proceeds to the next step. If the status of the inference service instance is error, it means that the service has failed to start and an error is returned. If the status of the inference service instance is pending, it means that the resources are full and waiting for available resources. If the status of the inference service instance is Notfound, it means that the current machine learning model has not been deployed and an error is returned.

[0023] When performing prediction processing, determine whether the machine learning model contains a Transformer component. If it does, it means that there is preprocessing logic in the online prediction. Set the Transformer component as the call address, perform data preprocessing first, and then send the preprocessing result to the predictor component. After that, perform postprocessing on the result returned by the predictor component, and finally return the result. Otherwise, set the Predictor component as the call address, directly perform prediction based on the original data and the machine learning model specified in the online prediction request, and finally return the result.

[0024] Secondly, the present invention provides a database-integrated machine learning online inference service system, which is applied to a client, a database and an online inference service, including a model and online prediction management module, a model permission management module, a database table design module and an online inference service management module;

[0025] The model and online prediction management module is used to perform the following: build built-in SQL functions in the database and define the definition method of SQL functions. Provide model management services and online prediction services through SQL functions. The model management service supports model import, model list viewing, model details viewing and model deletion. The online prediction service is used to verify the permissions of online prediction requests initiated by users through the client and push the online prediction requests that have passed the permission verification to the online inference service.

[0026] The model permission management module is used to perform the following: build the database built-in model permission levels and management functions and define the calling methods of the model permission levels and management functions. It provides model permission level management services through the model permission levels and management functions. The model permission level management service supports setting model permission levels, setting user roles and permissions, and managing model permissions. Among them, the model permission management supports users to grant, revoke and view permissions for machine learning models.

[0027] The database table design module is used to perform the following: build a model details table and a permission details table within the database. For machine learning models imported into the database, the model details table is used to store the model and model metadata, and the permission details table is used to store the model permission level of each machine learning model, the role of each user, and the permissions for the machine learning model.

[0028] The online inference service management module is used to perform the following: build an online inference service. The online inference service process includes a model deployment process and an online prediction process. The model deployment process includes storing and managing model-related files of the machine learning model specified in the online prediction request, and creating an inference service instance. The inference service instance is used to perform machine learning model deployment. The online prediction process includes detecting the status of the inference service instance and the prediction processing logic. The prediction processing logic is used to make predictions based on the machine learning model and return the prediction results to the database.

[0029] Preferably, the database and the online inference service communicate via the gRPC mechanism.

[0030] Preferably, the SQL functions in the database used to implement management services and online prediction services include the model list viewing function show_models, the model import function import_model, the model details viewing function show_model, the online prediction function predict, and the model deletion function remove_model;

[0031] Model permission levels and management functions include the model permission granting function `grant_privilege_on_model`, the model permission cancellation function `revoke_privilege_on_model`, and the model permission viewing function `has_model_privilege`.

[0032] Model permission levels include ALL, SELECT, CREATE, DELETE, and PREDICT. The built-in functions corresponding to the SELECT permission are Show_models and Show_model. Show_models is used to view the list of machine learning models, and Show_model is used to view the details of machine learning models. The built-in function corresponding to the CREATE permission is Import_model, which is used to import machine learning models. The built-in function corresponding to the DELETE permission is Remove_model, which is used to delete machine learning models. The built-in function corresponding to the Predict permission is PREDICT, which is used for online prediction.

[0033] User roles include system administrator and owner. The system administrator role has ALL permissions for all machine learning models by default, and the owner role has ALL permissions for importing machine learning models by default. System administrators and owners can grant machine learning model permissions to a user or role, but users cannot transfer their permissions to other users. System administrators and model creators have permission to call the model permission granting function and the model permission revocation function, while system administrators, model creators, and model users have permission to call the model permission viewing function.

[0034] Preferably, the online prediction service is used to perform the following permission verification for online prediction requests initiated by users through the client:

[0035] Verify if the current user is a system administrator. If so, pass the permission verification directly; otherwise, proceed to the next step.

[0036] Query the current user's role and the permission details table to verify whether the user's role and permissions match the current operation permission requirements. If they match, the permission verification passes; otherwise, the permission verification fails and the service is denied.

[0037] As a preferred option, an online inference service is built on KServe, and the online predictive analytics workload runs as a cloud service on the Kubernetes platform.

[0038] In the model deployment process, the model-related files of the machine learning model specified in the line prediction request are written to the local PVC shared file system of Kubernetes for unified management. For each stored machine learning model, a new folder is created and the folder name is used as a unique identifier. The folder name format is model name-model version. A single machine learning model folder has one or two subdirectories. The model subdirectory is required and is used to store model entities and static files. The data-preparation directory is optional and is used to store user preprocessing logic, processing logic and related dependencies.

[0039] Each inference service consists of a Predictor component and a Transformer component. The data preprocessing stage and the prediction result postprocessing stage are deployed in the Transformer component, and the model is deployed in the Predictor component. Each component is a container in Kubernetes. When it starts, it needs to load a base image and a directory mounted in the Kubernetes local PVC shared file system. When the Predictor component starts, it needs to mount the model directory under the local folder of the model to the fixed directory of the container. When the Transformer component starts, it needs to mount the data-preparation directory under the local folder of the machine learning model to the fixed directory of the container.

[0040] In the online prediction process, the status of the inference service instance is checked. If the status of the inference service instance is ready, it means that the service is available and proceeds to the next step. If the status of the inference service instance is error, it means that the service has failed to start and an error is returned. If the status of the inference service instance is pending, it means that the resources are full and waiting for available resources. If the status of the inference service instance is Notfound, it means that the current machine learning model has not been deployed and an error is returned.

[0041] When performing prediction processing, determine whether the machine learning model contains a Transformer component. If it does, it means that there is preprocessing logic in the online prediction. Set the Transformer component as the call address, perform data preprocessing first, and then send the preprocessing result to the predictor component. After that, perform postprocessing on the result returned by the predictor component, and finally return the result. Otherwise, set the Predictor component as the call address, directly perform prediction based on the original data and the machine learning model specified in the online prediction request, and finally return the result.

[0042] The method and system for database-integrated machine learning online inference service of the present invention have the following advantages: Through the design of components such as SQL built-in functions, permission management, related database tables and online inference internal interfaces, a database-integrated machine learning online inference service is built based on KServe service, enabling users to call machine learning model management and online inference capabilities in the form of SQL, and conveniently use machine learning related algorithms without additional learning of machine learning knowledge, which can expand the application scenarios of machine learning algorithms; at the same time, the close integration of model and data can also improve the training and inference efficiency of the model. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] The invention will be further described below with reference to the accompanying drawings.

[0045] Figure 1 This is a flowchart of a method for integrating a database with an online machine learning inference service, as described in Example 1.

[0046] Figure 2 This is a block diagram of the overall architecture of the database and online inference service in a method for integrating a database with a machine learning online inference service, as described in Example 1. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0048] This invention provides a method and system for integrating online inference services for machine learning with a database, which addresses the technical problem of how to combine machine learning models with a database and lower the barrier for users to use machine learning models.

[0049] Example 1:

[0050] This invention discloses a method for integrating a database with an online machine learning inference service. The method is used to construct an integrated system that includes a database and an online inference service. The integrated system works with the client to provide online inference services. The method includes four steps: model and online prediction management, model permission management, database table design, and online inference service management.

[0051] Step S100 Model and Online Prediction Management: Construct built-in SQL functions in the database and define the definition method of the SQL functions. Provide model management services and online prediction services through SQL functions. The model management service supports model import, model list viewing, model details viewing, and model deletion. The online prediction service is used to verify the permissions of online prediction requests initiated by users through the client and push online prediction requests that have passed the permission verification to the online inference service.

[0052] In this embodiment, the database and the online inference service communicate via the gRPC mechanism.

[0053] The SQL functions within the database used to implement management and online prediction services include: `show_models` for viewing the model list, `import_model` for importing models, `show_model` for viewing model details, `predict` for online prediction, and `remove_model` for deleting models. The function formats and SQL calling methods are as follows:

[0054] The function `show_models()` displays the list of models. The SQL syntax for calling it is: `SELECT * from show_models()`.

[0055] The model import function is represented as follows:

[0056] The import function has the following parameters: import_model(model_name, description, problem_type, framework, runtime, input_schema, output_schema, model_entity, transformer_entity, metrics). Table 1 shows the descriptions of each parameter.

[0057] Table 1. Description of Imported Function Parameters

[0058]

[0059]

[0060] The SQL call method is as follows:

[0061] SELECT import_model('Tom.PowerGen','A model to predict powergeneration','regression','tensorflow_2.8','python_3.8','[{"name":"current_phase_a verage","type":"float"},{"name":"weather_temperature_celsius","type":"float"}]','[{"name":"prediction","type":"float"}]',<model file> ,<transformer file> );

[0062] in,<model file> ,<transformer file> All data are binary.

[0063] The function to view model details is show_model(model_name), where model_name is the model name. The SQL call method is: SELECT * from show_model('Tom.PowerGen').

[0064] The online prediction function is `predict(columns) USING model_name FROM table_name`. The parameters of the online prediction function are explained in Table 2.

[0065] Table 2. Description of Online Prediction Function Parameters

[0066]

[0067] SQL calling method:

[0068] SELECT id, weather_temperature,

[0069] predict('current_phase_average','weather_tempe-rature_celsius','weather_relative_humidity','global_horizontal_radiation','diffuse_horizontal_radiation','wind_dir ection','weather_daily_rainfall') USING 'Tom.Power-Gen'FROM defaultdb.power_data.

[0070] The model deletion function is `remove_model(model_name)`, where `model_name` is the model name. The SQL calling method is as follows:

[0071] SELECT remove_model('Tom.PowerGen').

[0072] Step S200 Model Permission Management: Construct the database's built-in model permission levels and management functions, and define the calling methods for these functions. Provide model permission level management services through these functions. The model permission level management service supports setting model permission levels, user roles and permissions, and model permission management. Among these, model permission management supports granting, revoking, and viewing permissions for users to execute machine learning models.

[0073] In this embodiment, the model permission levels include ALL, SELECT, CREATE, DELETE, and PREDICT. The built-in functions corresponding to the SELECT permission include the Show_models function and the Show_model function. The Show_models function is used to view the list of machine learning models, and the Show_model function is used to view the details of machine learning models. The built-in function corresponding to the CREATE permission is the Import_model function, which is used to import machine learning models. The built-in function corresponding to the DELETE permission is the Remove_model function, which is used to delete machine learning models. The built-in function corresponding to the Predict permission is the PREDICT function, which is used for online prediction.

[0074] User roles include system administrator and owner. The system administrator role has ALL permissions for all machine learning models by default, and the owner has ALL permissions for importing machine learning models by default. System administrators and owners can grant machine learning model permissions to a user or role, but users cannot transfer their permissions to other users.

[0075] Model permission levels and management functions include the model permission granting function `grant_privilege_on_model`, the model permission cancellation function `revoke_privilege_on_model`, and the model permission viewing function `has_model_privilege`.

[0076] The model permission granting function is represented as follows:

[0077] The `grant_privilege_on_model(model_name,user_name|role_name,privilege)` function allows system administrators and model creators to call the model authorization function. The parameters of the function are described in Table 3.

[0078] Table 3. Description of Model Permission Granting Function Parameters

[0079]

[0080] The SQL call method is as follows:

[0081] SELECT grant_privilege_on_model('Tom.PowerGen','jerry',['delete','select']).

[0082] The function to revoke model permissions is expressed as follows:

[0083] revoke_privilege_on_model(model_name,user_name|role_name,privilege),

[0084] Roles authorized to call the model permission revocation function include system administrators and model creators. The meanings of the function parameters are the same as those of the authorization function parameters.

[0085] The SQL call method is as follows:

[0086] SELECT revoke_privilege_on_model('Tom.PowerGen','jerry',['delete','select']).

[0087] The function for viewing model permissions is as follows:

[0088] has_model_privilege(model_name,user_name|role_name,privilege),

[0089] Roles authorized to access the model access viewing function include system administrators, model creators, and model users. The meanings of the function parameters are the same as those of the authorization function parameters.

[0090] The SQL call method is as follows:

[0091] SELECT has_model_privilege('Tom.PowerGen','user01','SELECT').

[0092] The user permission verification steps during machine learning model management and online prediction capability invocation are as follows:

[0093] Verify if the current user is a system administrator. If so, pass the permission verification directly; otherwise, proceed to step 2.

[0094] Query the current user's role and the permissions table ML_PRIVILEGES. Verify whether the user's role and permissions match the current operation's permission requirements. If they match, the permission verification passes; otherwise, the permission verification fails and the service is denied.

[0095] Correspondingly, the online prediction service is used to perform the following permission verification for online prediction requests initiated by users through the client:

[0096] (1) Verify whether the current user is a system administrator. If yes, pass the permission verification directly. If not, proceed to the next step.

[0097] (2) Query the current user role and query the permission details table to verify whether the user role and permissions match the current operation permission requirements. If they match, the permission verification passes; otherwise, the permission verification fails and the service is denied.

[0098] Step S300 Database Table Design: Build a model details table and a permission details table within the database. For machine learning models imported into the database, the model details table is used to store the model and model metadata, and the permission details table is used to store the model permission level of each machine learning model, the role of each user, and the permissions for the machine learning model.

[0099] In this embodiment, two new system tables are added to the database: the model details table ML_MODELS and the permission details table ML_PRIVILEGES. The ML_MODELS table is used to store machine learning models and their related metadata, and the ML_PRIVILEGES table is used to store machine learning model permission information. Details of ML_PRIVILEGES are shown in Table 4, and details of ML_MODELS are shown in Table 5.

[0100] Table 4. Details of the ML_MODELS table

[0101]

[0102]

[0103] Table 5: Details of the ML_PRIVILEGES table

[0104]

[0105] Step S400 Online Inference Service Management: Build an online inference service. The online inference service process includes a model deployment process and an online prediction process. The model deployment process includes storing and managing model-related files of the machine learning model specified in the online prediction request, and creating an inference service instance. The inference service instance is used to execute machine learning model deployment. The online prediction process includes detecting the status of the inference service instance and prediction processing logic. The prediction processing logic is used to make predictions based on the machine learning model and return the prediction results to the database.

[0106] In the model deployment process, the model-related files of the machine learning model specified in the line prediction request are written to the local PVC shared file system of Kubernetes for unified management. For each stored machine learning model, a new folder is created and the folder name is used as a unique identifier. The folder name format is model name-model version. A single machine learning model folder has one or two subdirectories. The model subdirectory is required and is used to store model entities and static files. The data-preparation directory is optional and is used to store user preprocessing logic, processing logic and related dependencies.

[0107] Each inference service consists of a Predictor component and a Transformer component. The data preprocessing stage and the prediction result postprocessing stage are deployed in the Transformer component, while the model is deployed in the Predictor component. Each component is a container in Kubernetes. When starting, it needs to load a base image and a directory mounted in the Kubernetes local PVC shared file system. When the Predictor component starts, it needs to mount the model directory under the local folder of the model to the fixed directory of the container. When the Transformer component starts, it needs to mount the data-preparation directory under the local folder of the machine learning model to the fixed directory of the container.

[0108] In the online prediction process, the status of the inference service instance is checked. If the status of the inference service instance is ready, it means that the service is available and proceeds to the next step. If the status of the inference service instance is error, it means that the service has failed to start and an error is returned. If the status of the inference service instance is pending, it means that the resources are full and waiting for available resources. If the status of the inference service instance is Notfound, it means that the current machine learning model has not been deployed and an error is returned.

[0109] When performing prediction processing, determine whether the machine learning model contains a Transformer component. If it does, it means that there is preprocessing logic in the online prediction. Set the Transformer component as the call address, perform data preprocessing first, and then send the preprocessing result to the predictor component. After that, perform postprocessing on the result returned by the predictor component, and finally return the result. Otherwise, set the Predictor component as the call address, directly perform prediction based on the original data and the machine learning model specified in the online prediction request, and finally return the result.

[0110] In this embodiment, the online inference service is designed based on the open-source component KServe, leveraging KServe's excellent features such as providing a unified deployment and prediction call interface for models from various machine learning frameworks, and automatically monitoring and restarting deployed services to ensure high availability. The main processes of the online inference service include model deployment and online prediction.

[0111] Model Deployment Process: After receiving the model-related files, the model storage method is called to write the received compressed files to disk in the Kubernetes local PVC shared file system for unified management. For each stored model, a new folder is created, and the folder name is used as a unique identifier, following the naming convention of "model name-model version". A single model folder contains one or two subdirectories. The `model` subdirectory is mandatory and contains model entities and static files. The `data-preparation` directory is optional and contains user preprocessing logic, post-processing logic, and related dependencies.

[0112] Create an inference service instance that implements an inference service. Each inference service consists of a predictor component and a transform component. The data preprocessing stage and the prediction result post-processing stage are deployed in the transformer component, while the model is deployed in the predictor component. Each component is a container in Kubernetes. Upon startup, it needs to load a base image and mount directories from the Kubernetes local PVC shared file system. The base image is directly related to the algorithm framework used. When the predictor component starts, it needs to mount the `model` directory from the local folder corresponding to the model to a fixed directory in the container. When the transformer component starts, it needs to mount the `data-preparation` directory from the local folder corresponding to the model to a fixed directory in the container.

[0113] Check the status of the inference service instance. If the status is ready, the service is available and proceed to the next step. If the status is error, the service failed to start and an error is returned. If the status is pending, the system resources are full and waiting for available resources is required. If the status is Not found, the current model has not been deployed and an error is returned.

[0114] Determine if the model includes a Transformer. If it does, it indicates that online prediction involves preprocessing logic. Set the Transformer component as the calling address, perform data preprocessing first, then send the preprocessed results to the predictor component. Next, perform post-processing on the predictor component's return result, and finally return the result. Otherwise, set the Predictor component as the calling address, directly perform prediction based on the original data, and finally return the result.

[0115] Based on the method disclosed in this embodiment, an integrated system for database-integrated machine learning online inference services is constructed, the structure of which is as follows: Figure 2As shown in the diagram, the database stores metadata related to the machine learning model and the model itself, and includes built-in native predictive analytics functions. The online inference service is built on KServe, and the online predictive analytics workload runs as a cloud service on the Kubernetes platform. The database and the online inference service communicate via gRPC, employing a consistent security authentication method and access control system. When a user invokes the native predictive analytics functions via SQL on the client side, the database queries and retrieves its stored model and related data, then sends it to the online inference service for model deployment and online prediction.

[0116] The detailed operation process of the integrated system is as follows:

[0117] (1) Users can use SQL to call the show_models function to view the list of existing models in the database;

[0118] (2) Users can import machine learning models by calling the import_model function via SQL. The model and related metadata will be saved to the database system table ML_MODELS.

[0119] (3) For newly imported models, the database system will automatically deploy them to the online inference service. Specifically, after receiving the deployment request, the database will send the model to the online inference service via gRPC. The online inference service will check and deploy the model.

[0120] (4) Users initiate online model prediction requests by calling the predict function via SQL. After receiving the request, the database first retrieves the ML_PRIVILEGES table to verify user permissions. If the verification is successful, the database sends the model metadata to the online inference service via gRPC. The online inference service then calls the relevant interfaces to complete the online prediction.

[0121] (5) The online prediction results are returned to the database system via gRPC, and then further returned to the caller of the predict function.

[0122] Example 2:

[0123] This invention discloses a system for integrating a database with an online machine learning inference service. The system is used to construct an integrated system that includes a database and an online inference service. The integrated system works with the client to provide online inference services. The system includes a model and online prediction management module, a model permission management module, a database table design module, and an online inference service management module.

[0124] The model and online prediction management module is used to perform the following: build built-in SQL functions in the database and define the definition method of the SQL functions. The SQL functions provide model management services and online prediction services. The model management service supports model import, model list viewing, model details viewing, and model deletion. The online prediction service is used to verify the permissions of online prediction requests initiated by users through the client and push the online prediction requests that have passed the permission verification to the online inference service.

[0125] In this embodiment, the database and the online inference service communicate via the gRPC mechanism.

[0126] The SQL functions within the database used to implement management and online prediction services include: `show_models` for viewing the model list, `import_model` for importing models, `show_model` for viewing model details, `predict` for online prediction, and `remove_model` for deleting models. The function formats and SQL calling methods are as follows:

[0127] The function `show_models()` displays the list of models. The SQL syntax for calling it is: `SELECT * from show_models()`.

[0128] The model import function is represented as follows:

[0129] import_model(model_name, description, problem_type, framework, runtime, input_schema, output_schema, model_entity, transformer_entity, metrics).

[0130] The SQL call method is as follows:

[0131] SELECT import_model('Tom.PowerGen','A model to predict powergeneration','regression','tensorflow_2.8','python_3.8','[{"name":"current_phase_a verage","type":"float"},{"name":"weather_temperature_celsius","type":"float"}]','[{"name":"prediction","type":"float"}]',<model file> ,<transformer file> );

[0132] in,<model file> ,<transformer file> All data are binary.

[0133] The function to view model details is show_model(model_name), where model_name is the model name. The SQL call method is: SELECT * from show_model('Tom.PowerGen').

[0134] The online prediction function for the model is `predict(columns)` which uses `USING model_name FROM table_name`.

[0135] SQL calling method:

[0136] SELECT id, weather_temperature,

[0137] predict('current_phase_average','weather_tempe-rature_celsius','weather_relative_humidity','global_horizontal_radiation','diffuse_horizontal_radiation','wind_dir ection','weather_daily_rainfall') USING 'Tom.Power-Gen'FROM defaultdb.power_data.

[0138] The model deletion function is `remove_model(model_name)`, where `model_name` is the model name. The SQL calling method is as follows:

[0139] SELECT remove_model('Tom.PowerGen').

[0140] The model permission management module is used to perform the following: build the database's built-in model permission levels and management functions, define the calling methods of the model permission levels and management functions, provide model permission level management services through the model permission levels and management functions, and support model permission level settings, user role and permission settings, and model permission management. Among them, model permission management supports users to grant, revoke and view permissions for machine learning models.

[0141] In this embodiment, the model permission levels include ALL, SELECT, CREATE, DELETE, and PREDICT. The built-in functions corresponding to the SELECT permission include the Show_models function and the Show_model function. The Show_models function is used to view the list of machine learning models, and the Show_model function is used to view the details of machine learning models. The built-in function corresponding to the CREATE permission is the Import_model function, which is used to import machine learning models. The built-in function corresponding to the DELETE permission is the Remove_model function, which is used to delete machine learning models. The built-in function corresponding to the Predict permission is the PREDICT function, which is used for online prediction.

[0142] User roles include system administrator and owner. The system administrator role has ALL permissions for all machine learning models by default, and the owner has ALL permissions for importing machine learning models by default. System administrators and owners can grant machine learning model permissions to a user or role, but users cannot transfer their permissions to other users.

[0143] Model permission levels and management functions include the model permission granting function `grant_privilege_on_model`, the model permission cancellation function `revoke_privilege_on_model`, and the model permission viewing function `has_model_privilege`.

[0144] The model permission granting function is represented as follows:

[0145] The `grant_privilege_on_model(model_name, user_name|role_name, privilege)` function allows system administrators and model creators to call the model authorization function.

[0146] The SQL call method is as follows:

[0147] SELECT grant_privilege_on_model('Tom.PowerGen','jerry',['delete','select']).

[0148] The function to revoke model permissions is expressed as follows:

[0149] revoke_privilege_on_model(model_name,user_name|role_name,privilege),

[0150] Roles authorized to call the model permission revocation function include system administrators and model creators. The meanings of the function parameters are the same as those of the authorization function parameters.

[0151] The SQL call method is as follows:

[0152] SELECT revoke_privilege_on_model('Tom.PowerGen','jerry',['delete','select']).

[0153] The function for viewing model permissions is as follows:

[0154] has_model_privilege(model_name,user_name|role_name,privilege),

[0155] Roles authorized to access the model access viewing function include system administrators, model creators, and model users. The meanings of the function parameters are the same as those of the authorization function parameters.

[0156] The SQL call method is as follows:

[0157] SELECT has_model_privilege('Tom.PowerGen','user01','SELECT').

[0158] The user permission verification steps during machine learning model management and online prediction capability invocation are as follows:

[0159] Verify if the current user is a system administrator. If so, pass the permission verification directly; otherwise, proceed to step 2.

[0160] Query the current user's role and the permissions table ML_PRIVILEGES. Verify whether the user's role and permissions match the current operation's permission requirements. If they match, the permission verification passes; otherwise, the permission verification fails and the service is denied.

[0161] Correspondingly, the online prediction service is used to perform the following permission verification for online prediction requests initiated by users through the client:

[0162] (1) Verify whether the current user is a system administrator. If yes, pass the permission verification directly. If not, proceed to the next step.

[0163] (2) Query the current user role and query the permission details table to verify whether the user role and permissions match the current operation permission requirements. If they match, the permission verification passes; otherwise, the permission verification fails and the service is denied.

[0164] The database table design module is used to perform the following: build a model details table and a permission details table within the database. For machine learning models imported into the database, the model details table is used to store the model and model metadata, and the permission details table is used to store the model permission level of each machine learning model, the role of each user, and the permissions for the machine learning model.

[0165] In this embodiment, two new system tables are added to the database: the model details table ML_MODELS and the permission details table ML_PRIVILEGES. The ML_MODELS table is used to store machine learning models and their related metadata, and the ML_PRIVILEGES table is used to store machine learning model permission information.

[0166] The online inference service management module is used to perform the following: build an online inference service. The online inference service process includes a model deployment process and an online prediction process. The model deployment process includes storing and managing model-related files of the machine learning model specified in the online prediction request, and creating an inference service instance. The inference service instance is used to perform machine learning model deployment. The online prediction process includes detecting the status of the inference service instance and the prediction processing logic. The prediction processing logic is used to make predictions based on the machine learning model and return the prediction results to the database.

[0167] In the model deployment process, the model-related files of the machine learning model specified in the line prediction request are written to the local PVC shared file system of Kubernetes for unified management. For each stored machine learning model, a new folder is created and the folder name is used as a unique identifier. The folder name format is model name-model version. A single machine learning model folder has one or two subdirectories. The model subdirectory is required and is used to store model entities and static files. The data-preparation directory is optional and is used to store user preprocessing logic, processing logic and related dependencies.

[0168] Each inference service consists of a Predictor component and a Transformer component. The data preprocessing stage and the prediction result postprocessing stage are deployed in the Transformer component, while the model is deployed in the Predictor component. Each component is a container in Kubernetes. When starting, it needs to load a base image and a directory mounted in the Kubernetes local PVC shared file system. When the Predictor component starts, it needs to mount the model directory under the local folder of the model to the fixed directory of the container. When the Transformer component starts, it needs to mount the data-preparation directory under the local folder of the machine learning model to the fixed directory of the container.

[0169] In the online prediction process, the status of the inference service instance is checked. If the status of the inference service instance is ready, it means that the service is available and proceeds to the next step. If the status of the inference service instance is error, it means that the service has failed to start and an error is returned. If the status of the inference service instance is pending, it means that the resources are full and waiting for available resources. If the status of the inference service instance is Notfound, it means that the current machine learning model has not been deployed and an error is returned.

[0170] When performing prediction processing, determine whether the machine learning model contains a Transformer component. If it does, it means that there is preprocessing logic in the online prediction. Set the Transformer component as the call address, perform data preprocessing first, and then send the preprocessing result to the predictor component. After that, perform postprocessing on the result returned by the predictor component, and finally return the result. Otherwise, set the Predictor component as the call address, directly perform prediction based on the original data and the machine learning model specified in the online prediction request, and finally return the result.

[0171] In this embodiment, the online inference service is designed based on the open-source component KServe, leveraging KServe's excellent features such as providing a unified deployment and prediction call interface for models from various machine learning frameworks, and automatically monitoring and restarting deployed services to ensure high availability. The main processes of the online inference service include model deployment and online prediction.

[0172] Model Deployment Process: After receiving the model-related files, the model storage method is called to write the received compressed files to disk in the Kubernetes local PVC shared file system for unified management. For each stored model, a new folder is created, and the folder name is used as a unique identifier, following the naming convention of "model name-model version". A single model folder contains one or two subdirectories. The `model` subdirectory is mandatory and contains model entities and static files. The `data-preparation` directory is optional and contains user preprocessing logic, post-processing logic, and related dependencies.

[0173] Create an inference service instance that implements an inference service. Each inference service consists of a predictor component and a transform component. The data preprocessing stage and the prediction result post-processing stage are deployed in the transformer component, while the model is deployed in the predictor component. Each component is a container in Kubernetes. Upon startup, it needs to load a base image and mount directories from the Kubernetes local PVC shared file system. The base image is directly related to the algorithm framework used. When the predictor component starts, it needs to mount the `model` directory from the local folder corresponding to the model to a fixed directory in the container. When the transformer component starts, it needs to mount the `data-preparation` directory from the local folder corresponding to the model to a fixed directory in the container.

[0174] Check the status of the inference service instance. If the status is ready, the service is available and proceed to the next step. If the status is error, the service failed to start and an error is returned. If the status is pending, the system resources are full and waiting for available resources is required. If the status is Not found, the current model has not been deployed and an error is returned.

[0175] Determine if the model includes a Transformer. If it does, it indicates that online prediction involves preprocessing logic. Set the Transformer component as the calling address, perform data preprocessing first, then send the preprocessed results to the predictor component. Next, perform post-processing on the predictor component's return result, and finally return the result. Otherwise, set the Predictor component as the calling address, directly perform prediction based on the original data, and finally return the result.

[0176] Based on the system disclosed in this embodiment, an integrated system for database-integrated machine learning online inference services can be constructed using the method disclosed in Embodiment 1. Through this integrated system, users only need to master the database operation methods to use machine learning-related algorithms for predictive analysis.

[0177] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the means in the different embodiments described above, and these embodiments are also within the protection scope of the present invention.

Claims

1. A method for integrating a database with an online machine learning inference service, characterized in that, The method for constructing an integrated system that includes a database and an online inference service, wherein the integrated system works with a client to provide online inference services, includes the following steps: Model and Online Prediction Management: Construct built-in SQL functions in the database and define the definition method of SQL functions. Provide model management services and online prediction services through SQL functions. The model management service supports model import, model list viewing, model details viewing, and model deletion. The online prediction service is used to verify the permissions of online prediction requests initiated by users through the client and push the online prediction requests that have passed the permission verification to the online inference service. Model permission management: Construct database-built-in model permission levels and management functions, and define the calling methods of model permission levels and management functions. Provide model permission level management services through model permission levels and management functions. The model permission level management service supports setting model permission levels, setting user roles and permissions, and managing model permissions. Among them, model permission management supports granting, revoking and viewing permissions for users to execute machine learning models. Database table design: Build a model details table and a permission details table within the database. For machine learning models imported into the database, the model details table is used to store the model and model metadata, and the permission details table is used to store the model permission level of each machine learning model, the role of each user, and the permissions for the machine learning model. Online Inference Service Management: Building an online inference service. The online inference service process includes a model deployment process and an online prediction process. The model deployment process includes storing and managing model-related files of the machine learning model specified in the online prediction request, and creating an inference service instance. The inference service instance is used to execute machine learning model deployment. The online prediction process includes detecting the status of the inference service instance and the prediction processing logic. The prediction processing logic is used to make predictions based on the machine learning model and return the prediction results to the database. Each inference service consists of a Predictor component and a Transformer component. The data preprocessing stage and the prediction result postprocessing stage are deployed in the Transformer component, and the model is deployed in the Predictor component. Each component is a container in Kubernetes. When it starts, it needs to load a base image and a directory mounted in the Kubernetes local PVC shared file system. When the Predictor component starts, it needs to mount the model directory under the local folder of the model to the fixed directory of the container. When the Transformer component starts, it needs to mount the data-preparation directory under the local folder of the machine learning model to the fixed directory of the container. In the online prediction process, the status of the inference service instance is checked. If the inference service instance status is "ready", it means the service is available and proceeds to the next step. If the inference service instance status is "error", it means the service failed to start and an error is returned. If the inference service instance status is "pending", it means the resources are full and waiting for available resources. If the inference service instance status is "Notfound", it means the current machine learning model has not been deployed and an error is returned.

2. The method for integrating databases with online machine learning inference services according to claim 1, characterized in that, The database and online inference service communicate via the gRPC mechanism.

3. The method for database integration machine learning online inference service according to claim 1 or 2, characterized in that, The SQL functions within the database used to implement management and online prediction services include the model list viewing function `show_models`, the model import function `import_model`, the model details viewing function `show_model`, the online prediction function `predict`, and the model deletion function `remove_model`. Model permission levels and management functions include the model permission granting function `grant_privilege_on_model`, the model permission cancellation function `revoke_privilege_on_model`, and the model permission viewing function `has_model_privilege`. Model permission levels include ALL, SELECT, CREATE, DELETE, and PREDICT. The built-in functions corresponding to the SELECT permission are Show_models and Show_model. Show_models is used to view the list of machine learning models, and Show_model is used to view the details of machine learning models. The built-in function corresponding to the CREATE permission is Import_model, which is used to import machine learning models. The built-in function corresponding to the DELETE permission is Remove_model, which is used to delete machine learning models. The built-in function corresponding to the Predict permission is PREDICT, which is used for online prediction. User roles include system administrator and owner. The system administrator role has ALL permissions for all machine learning models by default, and the owner role has ALL permissions for importing machine learning models by default. System administrators and owners can grant machine learning model permissions to a user or role, but users cannot transfer their permissions to other users. System administrators and model creators have permission to call the model permission granting function and the model permission revocation function, while system administrators, model creators, and model users have permission to call the model permission viewing function.

4. The method for integrating databases with online machine learning inference services according to claim 3, characterized in that, The online prediction service is used to perform the following permission verification for online prediction requests initiated by users through the client: Verify if the current user is a system administrator. If so, pass the permission verification directly; otherwise, proceed to the next step. Query the current user's role and the permission details table to verify whether the user's role and permissions match the current operation permission requirements. If they match, the permission verification passes; otherwise, the permission verification fails and the service is denied.

5. The method for integrating databases with online machine learning inference services according to claim 1, characterized in that, An online inference service is built on KServe, and the online predictive analytics workload runs as a cloud service on the Kubernetes platform. In the model deployment process, the model-related files of the machine learning model specified in the line prediction request are written to the local PVC shared file system of Kubernetes for unified management. For each stored machine learning model, a new folder is created and the folder name is used as a unique identifier. The folder name format is model name-model version. A single machine learning model folder has one or two subdirectories. The model subdirectory is required and is used to store model entities and static files. The data-preparation directory is optional and is used to store user preprocessing logic, processing logic and related dependencies. When performing prediction processing, determine whether the machine learning model contains a Transformer component. If it does, it means that there is preprocessing logic in the online prediction. Set the Transformer component as the call address, perform data preprocessing first, and then send the preprocessing result to the predictor component. After that, perform postprocessing on the result returned by the predictor component, and finally return the result. Otherwise, set the Predictor component as the call address, directly perform prediction based on the original data and the machine learning model specified in the online prediction request, and finally return the result.

6. A system for integrating databases with online machine learning inference services, characterized in that, This system is used to build an integrated system that includes a database and online inference services. The integrated system works with clients to provide online inference services. The system includes a model and online prediction management module, a model permission management module, a database table design module, and an online inference service management module. The model and online prediction management module is used to perform the following: build built-in SQL functions in the database and define the definition method of SQL functions. Provide model management services and online prediction services through SQL functions. The model management service supports model import, model list viewing, model details viewing and model deletion. The online prediction service is used to verify the permissions of online prediction requests initiated by users through the client and push the online prediction requests that have passed the permission verification to the online inference service. The model permission management module is used to perform the following: build the database built-in model permission levels and management functions and define the calling methods of the model permission levels and management functions. It provides model permission level management services through the model permission levels and management functions. The model permission level management service supports setting model permission levels, setting user roles and permissions, and managing model permissions. Among them, the model permission management supports users to grant, revoke and view permissions for machine learning models. The database table design module is used to perform the following: build a model details table and a permission details table within the database. For machine learning models imported into the database, the model details table is used to store the model and model metadata, and the permission details table is used to store the model permission level of each machine learning model, the role of each user, and the permissions for the machine learning model. The online inference service management module is used to perform the following: build an online inference service. The online inference service process includes a model deployment process and an online prediction process. The model deployment process includes storing and managing model-related files of the machine learning model specified in the online prediction request, and creating an inference service instance. The inference service instance is used to perform machine learning model deployment. The online prediction process includes detecting the status of the inference service instance and prediction processing logic. The prediction processing logic is used to make predictions based on the machine learning model and return the prediction results to the database. Each inference service consists of a Predictor component and a Transformer component. The data preprocessing stage and the prediction result postprocessing stage are deployed in the Transformer component, and the model is deployed in the Predictor component. Each component is a container in Kubernetes. When it starts, it needs to load a base image and a directory mounted in the Kubernetes local PVC shared file system. When the Predictor component starts, it needs to mount the model directory under the local folder of the model to the fixed directory of the container. When the Transformer component starts, it needs to mount the data-preparation directory under the local folder of the machine learning model to the fixed directory of the container. In the online prediction process, the status of the inference service instance is checked. If the inference service instance status is "ready", it means the service is available and proceeds to the next step. If the inference service instance status is "error", it means the service failed to start and an error is returned. If the inference service instance status is "pending", it means the resources are full and waiting for available resources. If the inference service instance status is "Notfound", it means the current machine learning model has not been deployed and an error is returned.

7. The system for integrating database-driven machine learning online inference services according to claim 6, characterized in that, The database and online inference service communicate via the gRPC mechanism.

8. The system for database-integrated machine learning online inference services according to claim 6 or 7, characterized in that, The SQL functions within the database used to implement management and online prediction services include the model list viewing function `show_models`, the model import function `import_model`, the model details viewing function `show_model`, the online prediction function `predict`, and the model deletion function `remove_model`. Model permission levels and management functions include the model permission granting function `grant_privilege_on_model`, the model permission cancellation function `revoke_privilege_on_model`, and the model permission viewing function `has_model_privilege`. Model permission levels include ALL, SELECT, CREATE, DELETE, and PREDICT. The built-in functions corresponding to the SELECT permission are Show_models and Show_model. Show_models is used to view the list of machine learning models, and Show_model is used to view the details of machine learning models. The built-in function corresponding to the CREATE permission is Import_model, which is used to import machine learning models. The built-in function corresponding to the DELETE permission is Remove_model, which is used to delete machine learning models. The built-in function corresponding to the Predict permission is PREDICT, which is used for online prediction. User roles include system administrator and owner. The system administrator role has ALL permissions for all machine learning models by default, and the owner role has ALL permissions for importing machine learning models by default. System administrators and owners can grant machine learning model permissions to a user or role, but users cannot transfer their permissions to other users. System administrators and model creators have permission to call the model permission granting function and the model permission revocation function, while system administrators, model creators, and model users have permission to call the model permission viewing function.

9. The system for integrating database-driven machine learning online inference services according to claim 8, characterized in that, The online prediction service is used to perform the following permission verification for online prediction requests initiated by users through the client: Verify if the current user is a system administrator. If so, pass the permission verification directly; otherwise, proceed to the next step. Query the current user's role and the permission details table to verify whether the user's role and permissions match the current operation permission requirements. If they match, the permission verification passes; otherwise, the permission verification fails and the service is denied.

10. The system for integrating database-driven machine learning online inference services according to claim 8, characterized in that, An online inference service is built on KServe, and the online predictive analytics workload runs as a cloud service on the Kubernetes platform. In the model deployment process, the model-related files of the machine learning model specified in the line prediction request are written to the local PVC shared file system of Kubernetes for unified management. For each stored machine learning model, a new folder is created and the folder name is used as a unique identifier. The folder name format is model name-model version. A single machine learning model folder has one or two subdirectories. The model subdirectory is required and is used to store model entities and static files. The data-preparation directory is optional and is used to store user preprocessing logic, processing logic and related dependencies. When performing prediction processing, determine whether the machine learning model contains a Transformer component. If it does, it means that there is preprocessing logic in the online prediction. Set the Transformer component as the call address, perform data preprocessing first, and then send the preprocessing result to the predictor component. After that, perform postprocessing on the result returned by the predictor component, and finally return the result. Otherwise, set the Predictor component as the call address, directly perform prediction based on the original data and the machine learning model specified in the online prediction request, and finally return the result.

Images