Model deployment method and device, electronic equipment and computer readable storage medium
By integrating SQL statements of the integration model into a file and deploying them to the target data platform in the production environment, the problem of high dependence on the integration model deployment solution is solved, and efficient and low-cost deployment and inference effects are achieved.
Patent Information
- Application Number
- CN202411934024.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-25
AI Technical Summary
The existing integrated model deployment solutions depend heavily on the production environment and the model development environment, resulting in high deployment costs and difficult implementation in high-time and strong regulatory scenarios.
By obtaining the metadata information of the integration model and the tree structure information of the decision tree, corresponding SQL statements are generated, and these SQL statements are integrated into a file and deployed to the target data platform of the production environment.
It realizes the deployment and use of integrated models in a production environment without relying on the model development environment, reduces deployment costs, and reasoning results are consistent with the native model development environment in high-time and strong regulatory scenarios.
Smart Images
Figure CN119883290A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a model deployment method, device, electronic device, and computer-readable storage medium. Background Art
[0002] The Tree-based Ensemble Learning model based on the LightGBM (Light Gradient Boosting Machine) and XGboost (eXtremeGradient Boosting) algorithms has many characteristics such as fast fitting, distributed training, high fitting performance and high stability, and has become the preferred machine learning model for tabular data in many business scenarios.
[0003] Currently, there are usually two ways to deploy integrated models. The first is to perform model reasoning based on external environments such as Python / Java / C, and provide model prediction values to downstream tasks in the form of API interfaces through the scheduler. The second is to translate the model into a PMML (Predictive Model Markup Language) file, and load the PMML file of the trained integrated model through the execution engine in the background of the business system for model reasoning.
[0004] Among them, the model side and data side of the first deployment solution are usually located in different production environments, and additional data exchange resources and scheduling management modules are required to support the entire model reasoning process. In some application scenarios, the entire data transmission process will also be subject to internal compliance supervision.
[0005] The second deployment solution is mainly limited by the development resources and production cycle of the business system. The long change cycle is easy to miss the business development window, which goes against the original intention of supporting business development. In addition, the PMML files of the second deployment solution are generally deployed at the business layer. When the business layer calls the interface service of the PMML file of the data layer, the data needs to be transmitted across layers. In particular, in high-timeliness scenarios and strong regulatory scenarios, the above technical solutions are not easy to implement. Summary of the invention
[0006] The embodiments of the present application provide a model deployment method, device, electronic device and computer-readable storage medium to avoid the dependence of the production environment on the model development environment and reduce the cost of model deployment.
[0007] The present application embodiment adopts the following technical solutions:
[0008] In a first aspect, an embodiment of the present application provides a model deployment method, the method comprising:
[0009] Obtaining an integrated model to be deployed, wherein the integrated model includes multiple decision trees;
[0010] Acquire metadata information of the integrated model and tree structure information of the decision tree;
[0011] Acquire the SQL statement corresponding to the decision tree according to the tree structure information;
[0012] Integrate the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and obtain an SQL statement integration file corresponding to the integrated model;
[0013] Deploy the SQL statement integration file corresponding to the integration model to the target data platform of the production environment.
[0014] Optionally, the integrated model to be deployed is an integrated model described in a preset programming language, and metadata information of the integrated model is obtained by the following steps:
[0015] The integrated model is parsed based on the custom function of the preset programming language to obtain the metadata information, where the metadata information at least includes the model type, the objective function and the number of decision trees.
[0016] Optionally, the tree structure information of the decision tree sub-model is obtained by the following steps:
[0017] Traversing the decision tree by a preset traversal algorithm to obtain the node type of the current node, wherein the node type includes a root node, an internal node, and a leaf node;
[0018] Acquire node topology information of the current node according to the node type of the current node, wherein the node topology information includes decision condition information, lower-level node information, and predicted value;
[0019] The node type and node topology information of each node are stored through a custom data structure.
[0020] Optionally, acquiring a SQL statement corresponding to the decision tree sub-model according to the tree structure information includes:
[0021] Acquire node path information and path description rules corresponding to the decision tree according to the tree structure information;
[0022] Generate a set of SQL statement blocks corresponding to the decision tree according to the path description rule;
[0023] The SQL statement corresponding to the decision tree sub-model is obtained by combining the SQL statement block set.
[0024] Optionally, acquiring the node path information and path description rule corresponding to the decision tree according to the tree structure information includes:
[0025] Recursively searching the custom data structure using a preset traversal algorithm to obtain node path information of the decision tree;
[0026] The path description rule corresponding to the node path is obtained according to the node type, decision condition information and subordinate node information of each node in the node path.
[0027] Optionally, the metadata information includes at least a model type, an objective function, and the number of decision trees, and the integrating and processing the SQL statements of the multiple decision trees according to the metadata information corresponding to the integrated model includes:
[0028] Determine the decision tree combination algorithm of the integrated model according to the model type, objective function and number of decision trees;
[0029] The decision tree combination algorithm is used to integrate the SQL statements of multiple decision trees to obtain an SQL statement integration file corresponding to the integrated model.
[0030] In a second aspect, an embodiment of the present application further provides a model deployment device, the device comprising:
[0031] A model acquisition unit, used to acquire an integrated model to be deployed, wherein the integrated model includes multiple decision trees;
[0032] A model parsing unit, used to obtain metadata information of the integrated model and tree structure information of the decision tree;
[0033] An SQL statement generating unit, configured to obtain an SQL statement corresponding to the decision tree according to the tree structure information;
[0034] An SQL statement processing unit, configured to integrate the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and obtain an SQL statement integration file corresponding to the integrated model;
[0035] The SQL file deployment unit is used to deploy the SQL statement integration file corresponding to the integration model to the target data platform of the production environment.
[0036] Optionally, the SQL statement generation unit is specifically used to obtain the node path information and path description rules corresponding to the decision tree according to the tree structure information; generate a set of SQL statement blocks corresponding to the decision tree according to the path description rules; and obtain the SQL statement corresponding to the decision tree submodel by combining the SQL statement block set.
[0037] In a third aspect, an embodiment of the present application further provides an electronic device, including:
[0038] A processor; and a memory arranged to store computer executable instructions, which when executed cause the processor to perform a model deployment method.
[0039] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs, and when the one or more programs are executed by an electronic device including multiple applications, the electronic device executes a model deployment method.
[0040] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:
[0041] The embodiment of the present application first obtains the metadata information of the integrated model and the tree structure information of multiple decision trees of the integrated model, obtains the SQL statement corresponding to the decision tree according to the tree structure information, and integrates the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and deploys the SQL statement integration file corresponding to the integrated model to the target data platform in the production environment. In this way, when predicting data, the target data platform only needs to call the SQL statement integration file to efficiently obtain the equivalent SQL code file corresponding to the trained integrated model with a unified interface, which solves the problem that complex integrated models cannot directly predict data in the SQL environment, and the embodiment of the present application uses the SQL statement integration file directly in the database production environment, and its reasoning results are consistent with the reasoning results in the native model development environment, reducing the intermediate cross-platform costs, scheduling costs and data transmission costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0043] Figure 1 This is a schematic diagram of an implementation environment of a solution in the embodiments of the present application;
[0044] Figure 2 A schematic diagram of the main process of a model deployment in an embodiment of the present application;
[0045] Figure 3 A schematic diagram of a flow chart of a model deployment method in an embodiment of the present application;
[0046] Figure 4 This is a schematic diagram of a decision tree in an embodiment of the present application;
[0047] Figure 5This is a schematic diagram of an SQL code in an embodiment of the present application;
[0048] Figure 6 This is a schematic diagram of the structure of a model deployment device in an embodiment of the present application;
[0049] Figure 7 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0050] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0051] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0052] like Figure 1 As shown, a schematic diagram of an implementation environment of a solution in an embodiment of the present application is provided. The implementation environment of the model deployment solution of the present application includes a model development environment and a production environment.
[0053] The model development environment of this embodiment can be a server. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.
[0054] The production environment generally includes a data platform and a business platform. The data platform usually includes multiple databases or data warehouses and other data storage devices. The data platform also includes a data production platform for managing data storage devices. The databases here include, for example, PostgreSQL, Mysql, Hive and other databases.
[0055] In some embodiments of the present application, the model development environment is used to build and train the integrated model, and compile and save the trained integrated model. Optionally, the integrated model can be an integrated model described in a preset programming language and saved in a binary form in the model development environment. The preset programming language is, for example, the Python programming language, and the model development environment is a Python development environment.
[0056] The execution subject (such as a computing device) of the model deployment method translates the integrated model in the model development environment into an SQL statement integration file. Figure 2 As shown, the execution entity obtains the integration model to be deployed from the model development environment, parses the model to extract the path description rules, and then generates an SQL statement integration file corresponding to the integration model according to the path description rules, and sends the SQL statement integration file to the data production platform, so that the data production platform completes the model deployment based on the SQL statement integration file, and performs prediction processing on the prediction data in the data storage device based on the SQL statement integration file to obtain the final prediction result.
[0057] The present application embodiment provides a model deployment method, such as Figure 3 As shown, a flow chart of a model deployment method in an embodiment of the present application is provided, and the method at least includes the following steps S310 to S350:
[0058] Step S310, obtaining an integrated model to be deployed, wherein the integrated model includes multiple decision trees.
[0059] In some embodiments, the execution subject (e.g., computing device) of the model deployment method can obtain the integrated model through a wired connection or a wireless connection. The integrated model is a complex model constructed based on an integrated learning algorithm by setting multiple decision trees (also known as base learners) to minimize the prediction error. The integrated model is, for example, an integrated model based on algorithms such as LightGBM and XGboost.
[0060] It should be noted that the integrated model of the embodiment of the present application is a model that has been trained and described in a preset programming language.
[0061] It is understandable that the above computing device can be hardware or software. When the computing device is hardware, it can be a distributed cluster consisting of multiple servers or terminal devices, or a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. The computing device can also be multiple software or software modules used to provide distributed services, or it can be implemented as a single software or software module, which is not specifically limited here.
[0062] Step S320, obtaining metadata information of the integrated model and tree structure information of the decision tree.
[0063] The metadata information of the embodiment of the present application can be understood as the model description information of the integrated model, such as model type, objective function, number of decision trees and model parameters. The tree structure information of the decision tree can be understood as the description information of the decision tree, such as node type, node topology, decision condition information, etc.
[0064] When the execution subject of the embodiment of the present application obtains the integrated model to be deployed, it usually saves the integrated model in a binary cache file. In this way, the integrated model described in a preset programming language can be obtained by loading the binary cache file, and the metadata information of the integrated model can be obtained by parsing and identifying the integrated model. The tree structure information can also be obtained by traversing the decision tree of the integrated model.
[0065] Step S330: Obtain the SQL statement corresponding to the decision tree according to the tree structure information.
[0066] According to the tree structure information of the decision tree, the node path included in the decision tree can be determined. Figure 4 As shown, Figure 4 The decision tree in includes eight node paths, each node path corresponds to a data processing rule, and the SQL statement corresponding to each node path can be generated according to the data processing rule corresponding to each node path. By combining the SQL statements corresponding to the eight node paths, the SQL statement corresponding to the decision tree can be obtained.
[0067] Step S340, integrating the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and obtaining the SQL statement integration file corresponding to the integrated model.
[0068] As described in the aforementioned steps, the metadata information of the embodiment of the present application can be understood as the model description information of the integrated model. The combination algorithm of multiple decision trees can be determined based on the metadata information, so that the SQL statements of multiple decision trees can be integrated and processed based on the metadata information.
[0069] Step S350, deploying the SQL statement integration file corresponding to the integration model to the target data platform of the production environment.
[0070] The embodiment of the present application directly deploys the SQL statement integration file on the target data platform of the production environment. The target data platform can directly call the interface of the SQL statement integration file to predict the data of the target data platform and obtain the prediction result.
[0071] like Figure 3It can be seen from the model deployment method shown that the embodiment of the present application first obtains the metadata information of the integrated model and the tree structure information of multiple decision trees of the integrated model, obtains the SQL statement corresponding to the decision tree according to the tree structure information, and integrates the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and deploys the SQL statement integration file corresponding to the integrated model to the target data platform of the production environment. In this way, when predicting data, the target data platform only needs to call the SQL statement integration file to efficiently obtain the equivalent SQL code file corresponding to the trained integrated model with a unified interface, without the need for the target data platform to support model development environments such as Python and Java, which solves the problem that complex integrated models cannot directly predict data in an SQL environment, and the embodiment of the present application uses the SQL statement integration file directly in the database production environment, and its inference results are consistent with the inference results in the native model development environment, reducing the intermediate cross-platform costs, scheduling costs, and data transmission costs.
[0072] In some embodiments of the present application, the above step S310 of obtaining the integrated model to be deployed includes:
[0073] Load the integrated model cached in the binary cache file, obtain the integrated model described in a preset programming language, and load the integrated model in binary form into a Python object form.
[0074] When the executor of the model deployment method of this embodiment obtains the integrated model to be deployed from the model development environment, the integrated model is first cached into a binary cache file, and the Python object of the integrated model can be obtained by loading the binary cache file of the integrated model.
[0075] Accordingly, in some embodiments of the present application, the metadata information of the integrated model may be obtained through the following steps:
[0076] The integrated model is parsed based on the custom function of the preset programming language to obtain the metadata information, where the metadata information at least includes the model type, the objective function and the number of decision trees.
[0077] Assume that the model development environment is to build custom functions in Python programming language, use custom functions to develop integrated models, and save the integrated models in the model development environment after training. In the model deployment stage, when the executor of the model deployment method obtains the integrated model to be deployed in the model development environment, it loads the integrated model into Python object form and parses the integrated model through custom functions to obtain metadata information such as model type, objective function, and number of decision trees.
[0078] Among them, the model type refers to the machine learning algorithm used by the integrated model. The model type of this embodiment is, for example, an integrated model based on the LightGBM algorithm and the XGboost algorithm. The objective function is related to the model task. The task type of the integrated model can be obtained through the objective function, for example, to determine whether the integrated model is used to perform regression tasks or other types of tasks. The number of decision trees refers to the number of base learners of the integrated model. The metadata information of the embodiment of the present application is used to characterize the integration rules of the SQL statements of multiple decision trees, and the SQL statements of multiple decision trees can be integrated based on the metadata information.
[0079] In some embodiments of the present application, the tree structure information of the decision tree sub-model may be obtained by the following steps:
[0080] Traversing the decision tree by a preset traversal algorithm to obtain the node type of the current node, wherein the node type includes a root node, an internal node, and a leaf node;
[0081] Acquire node topology information of the current node according to the node type of the current node, wherein the node topology information includes decision condition information, lower-level node information, and predicted value;
[0082] The node type and node topology information of each node are stored through a custom data structure.
[0083] by Figure 4 Taking the decision tree shown in the figure as an example, a decision tree usually includes a root node (Column_8≤0.000), internal nodes (nodes other than the root node and leaf nodes) and leaf nodes (leaf0~leaf7). Each decision tree is traversed by a traversal algorithm such as Depth First Search (DFS). During the traversal process, the node type of the current node can be obtained according to the upper and lower node information of the current node. For example, the root node does not include the upper node, while the internal node includes the upper and lower nodes, and the leaf node only includes the upper node and does not have the lower node. If the current node is a root node and an internal node, the next level node information of the current node and its own decision condition information are extracted, for example Figure 4 The decision condition of the root node in is "Column_8≤0.000 (is the value of column 8 less than or equal to 0.000)", at this time, the node information of the next level of the current node and its own decision condition information are used as the node topology information of the current node. If the current node is a leaf node, the predicted value of the current node is extracted, for example Figure 4The predicted value of leaf node leaf0 in is 147.893. At this time, the predicted value of the current node is used as the node topology information of the current node. In other words, if the current node is a root node and an internal node, the predicted value in the node topology information of the current node corresponds to a null value (NULL). If the current node is a leaf node, the next-level node information and decision condition information in the node topology information of the current node are both null values.
[0084] In some embodiments of the present application, obtaining the SQL statement corresponding to the decision tree sub-model according to the tree structure information in the above step S330 specifically includes:
[0085] Acquire node path information and path description rules corresponding to the decision tree according to the tree structure information;
[0086] Generate a set of SQL statement blocks corresponding to the decision tree according to the path description rule;
[0087] The SQL statement corresponding to the decision tree sub-model is obtained by combining the SQL statement block set.
[0088] In some possible implementation schemes of this embodiment, obtaining the node path information and path description rules corresponding to the decision tree according to the tree structure information specifically includes:
[0089] Recursively searching the custom data structure using a preset traversal algorithm to obtain node path information of the decision tree;
[0090] The path description rule corresponding to the node path is obtained according to the node type, decision condition information and subordinate node information of each node in the node path.
[0091] For example, through the DFS algorithm, recursively search all paths of the decision tree. When traversing to a leaf node, the current path is stored as a complete node path in the path set corresponding to the decision tree. When traversing to an internal node, continue to traverse downward until all leaf nodes are completed. Figure 4 Take the decision tree shown as an example, Figure 4The decision tree in includes eight node paths. Taking the first node path (Column_8≤0.000→Column_2≤0.006→Column_6≤0.021→Leaf0:147.893) as an example, its corresponding path description rule is "When the value of Column_8 is less than or equal to 0.000, the value of Column_2 is less than or equal to 0.006, and the value of Column_6 is less than or equal to 0.021, the predicted value is 147.893". Based on this path description rule, SQL statement blocks can be automatically generated. In this way, the SQL statement block set corresponding to each decision tree can be obtained. All SQL statement blocks in the SQL statement block set are combined and processed by the "AND" and "OR" operators to obtain the SQL statement corresponding to each decision tree.
[0092] Of course, in other embodiments of the present application, the SQL statement block can also be adjusted, for example, referring to Figure 5 As shown, the adjusted statement blocks can also be combined into equivalent SQL statements through the "When" and "Then" operators.
[0093] After obtaining the SQL statements of multiple decision trees, the SQL statements of multiple decision trees can be integrated according to the metadata information corresponding to the integrated model. For example, if the task type of the integrated model is regression, the decision tree synthesis algorithm of the integrated model is to add the leaf node prediction values of each decision tree. If the task type of the integrated model is classification, the decision tree synthesis algorithm of the integrated model is to convert the prediction value into a numerical value through the Sigmoid function. In this way, the SQL statements of multiple decision trees are integrated accordingly through the decision tree synthesis algorithm of the integrated model, and the final SQL statement integration file can be obtained.
[0094] Based on the above embodiments, it can be seen that the model deployment method of the embodiments of the present application has at least the following advantages:
[0095] First, when predicting data, the target data platform only needs to call the SQL statement integration file to efficiently obtain the equivalent SQL code corresponding to the trained integration model through a unified interface, solving the problem that complex integration models cannot directly perform data prediction in the SQL environment;
[0096] Second, the SQL statement integration file is used directly in the database production environment. The inference results are consistent with those in the native model development environment, reducing the intermediate cross-platform costs, scheduling costs, and data transmission costs.
[0097] Third, in view of the structural differences and model information storage differences caused by different fitting strategies and optimization strategies of different integrated models, the embodiments of the present application reduce the deployment complexity caused by the above differences through a unified model deployment solution.
[0098] The present application embodiment also provides a model deployment device 600, such as Figure 6 As shown, a schematic diagram of the structure of a model deployment device in an embodiment of the present application is provided, wherein the device 600 includes: a model acquisition unit 610, a model parsing unit 620, an SQL statement generation unit 630, an SQL statement processing unit 640 and an SQL file deployment unit 650, wherein:
[0099] A model acquisition unit 610 is used to acquire an integrated model to be deployed, wherein the integrated model includes multiple decision trees;
[0100] A model parsing unit 620, configured to obtain metadata information of the integrated model and tree structure information of the decision tree;
[0101] An SQL statement generating unit 630 is used to obtain an SQL statement corresponding to the decision tree according to the tree structure information;
[0102] The SQL statement processing unit 640 is used to integrate the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and obtain the SQL statement integration file corresponding to the integrated model;
[0103] The SQL file deployment unit 650 is used to deploy the SQL statement integration file corresponding to the integration model to the target data platform of the production environment.
[0104] In some embodiments of the present application, the integrated model to be deployed is an integrated model described based on a preset programming language, and the model parsing unit 620 includes a first parsing module;
[0105] The first parsing module is used to parse the integrated model based on the custom function of the preset programming language to obtain the metadata information, where the metadata information at least includes the model type, the objective function and the number of decision trees.
[0106] In some embodiments of the present application, the model parsing unit 620 further includes a second parsing module;
[0107] The second parsing module is used to traverse the decision tree through a preset traversal algorithm to obtain the node type of the current node, and the node types include root nodes, internal nodes and leaf nodes; obtain the node topology information of the current node according to the node type of the current node, and the node topology information includes decision condition information, lower-level node information and predicted values; store the node type and node topology information of each node through a custom data structure.
[0108] In some embodiments of the present application, the SQL statement generation unit 630 is specifically used to obtain the node path information and path description rules corresponding to the decision tree according to the tree structure information; generate a set of SQL statement blocks corresponding to the decision tree according to the path description rules; and obtain the SQL statement corresponding to the decision tree submodel by combining the SQL statement block set.
[0109] In some embodiments of the present application, the SQL statement generation unit 630 is further used to recursively search the custom data structure through a preset traversal algorithm to obtain node path information of the decision tree; and obtain the path description rule corresponding to the node path according to the node type, decision condition information and subordinate node information of each node in the node path.
[0110] In some embodiments of the present application, the metadata information includes at least a model type, an objective function, and the number of decision trees. The SQL statement processing unit 640 is specifically used to determine the decision tree combination algorithm of the integrated model according to the model type, the objective function, and the number of decision trees; the SQL statements of multiple decision trees are integrated and processed through the decision tree combination algorithm to obtain a SQL statement integration file corresponding to the integrated model.
[0111] It can be understood that the above-mentioned model deployment device can implement the various steps of the model deployment method provided in the aforementioned embodiments. The relevant explanations about the model deployment method are applicable to the model deployment device and will not be repeated here.
[0112] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 7 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.
[0113] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0114] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.
[0115] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a model deployment device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:
[0116] Obtaining an integrated model to be deployed, wherein the integrated model includes multiple decision trees;
[0117] Acquire metadata information of the integrated model and tree structure information of the decision tree;
[0118] Acquire the SQL statement corresponding to the decision tree according to the tree structure information;
[0119] Integrate the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and obtain an SQL statement integration file corresponding to the integrated model;
[0120] Deploy the SQL statement integration file corresponding to the integration model to the target data platform of the production environment.
[0121] The above application Figure 3The method performed by the model deployment device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above model deployment method in combination with its hardware.
[0122] The electronic device may also perform Figure 3 The method executed by the model deployment device in Figure 3 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.
[0123] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 3 The method executed by the model deployment device in the illustrated embodiment is specifically used to execute:
[0124] Obtaining an integrated model to be deployed, wherein the integrated model includes multiple decision trees;
[0125] Acquire metadata information of the integrated model and tree structure information of the decision tree;
[0126] Acquire the SQL statement corresponding to the decision tree according to the tree structure information;
[0127] Integrate the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and obtain an SQL statement integration file corresponding to the integrated model;
[0128] Deploy the SQL statement integration file corresponding to the integration model to the target data platform of the production environment.
[0129] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0130] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0131] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0134] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0135] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0136] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0138] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A model deployment method, characterized in that: The model deployment method comprises: Obtaining an integrated model to be deployed, wherein the integrated model includes multiple decision trees; Acquire metadata information of the integrated model and tree structure information of the decision tree; Acquire the SQL statement corresponding to the decision tree according to the tree structure information; Integrate the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and obtain an SQL statement integration file corresponding to the integrated model; Deploy the SQL statement integration file corresponding to the integration model to the target data platform of the production environment.
2. The model deployment method according to claim 1, characterized in that: The integration model to be deployed is an integration model described in a preset programming language, and the metadata information of the integration model is obtained through the following steps: The integrated model is parsed based on the custom function of the preset programming language to obtain the metadata information, where the metadata information at least includes the model type, the objective function and the number of decision trees.
3. The model deployment method according to claim 1, characterized in that: The tree structure information of the decision tree sub-model is obtained through the following steps: Traversing the decision tree by a preset traversal algorithm to obtain the node type of the current node, wherein the node type includes a root node, an internal node, and a leaf node; Acquire node topology information of the current node according to the node type of the current node, wherein the node topology information includes decision condition information, lower-level node information, and predicted value; The node type and node topology information of each node are stored through a custom data structure.
4. The model deployment method according to claim 3, characterized in that: The step of obtaining the SQL statement corresponding to the decision tree sub-model according to the tree structure information includes: Acquire node path information and path description rules corresponding to the decision tree according to the tree structure information; Generate a set of SQL statement blocks corresponding to the decision tree according to the path description rule; The SQL statement corresponding to the decision tree sub-model is obtained by combining the SQL statement block set.
5. The model deployment method according to claim 4, characterized in that: The obtaining the node path information and path description rules corresponding to the decision tree according to the tree structure information includes: Recursively searching the custom data structure using a preset traversal algorithm to obtain node path information of the decision tree; The path description rule corresponding to the node path is obtained according to the node type, decision condition information and subordinate node information of each node in the node path.
6. The model deployment method according to claim 1, characterized in that: The metadata information at least includes a model type, an objective function, and the number of decision trees. The integrating and processing of the SQL statements of the multiple decision trees according to the metadata information corresponding to the integrated model includes: Determine the decision tree combination algorithm of the integrated model according to the model type, objective function and number of decision trees; The decision tree combination algorithm is used to integrate the SQL statements of multiple decision trees to obtain an SQL statement integration file corresponding to the integrated model.
7. A model deployment device, characterized in that: The model deployment device comprises: A model acquisition unit, used to acquire an integrated model to be deployed, wherein the integrated model includes multiple decision trees; A model parsing unit, used to obtain metadata information of the integrated model and tree structure information of the decision tree; An SQL statement generating unit, configured to obtain an SQL statement corresponding to the decision tree according to the tree structure information; An SQL statement processing unit, configured to integrate the SQL statements of multiple decision trees according to the metadata information corresponding to the integrated model, and obtain an SQL statement integration file corresponding to the integrated model; The SQL file deployment unit is used to deploy the SQL statement integration file corresponding to the integration model to the target data platform of the production environment.
8. The model deployment device according to claim 7, characterized in that: The SQL statement generation unit is specifically used to obtain the node path information and path description rules corresponding to the decision tree according to the tree structure information; generate a set of SQL statement blocks corresponding to the decision tree according to the path description rules; and obtain the SQL statement corresponding to the decision tree sub-model by combining the SQL statement block set.
9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the model deployment method according to any one of claims 1 to 6.
10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the model deployment method according to any one of claims 1 to 6.