Automated machine learning model deployment method and system

Abstract

An automated machine learning model deployment method and system is provided. First, a resource usage specification and a code module usage specification for the model are identified using exported data of the machine learning model and a model training environment specification. A code module installation specification is determined from the code module requirement specification and the target execution environment specification, and the code modules specified by the code module installation specification are installed in the target execution environment. The updated target execution environment is validated for execution of the model using updated target execution environment data. Execution of the model in the updated target execution environment is simulated. The model is deployed in the updated target execution environment in response to successful simulation.

Description

[Background technology]

[0001] The present invention relates generally to methods, systems, and computer program products for software deployment, and more particularly to methods, systems, and computer program products for automated machine learning model deployment.

[0002] A machine learning model is a model that learns to make predictions from input data. Typically, a machine learning model is first trained by applying one or more learning techniques to training data. Once the model meets one or more acceptance criteria (e.g., the accuracy of the model's output exceeds a threshold percentage), the model is considered sufficiently trained to be used for its intended purpose on live (non-training) data. However, model training may continue after the model is deployed in production. A machine learning model that operates on an incoming stream of data is also referred to as a machine learning pipeline or simply a pipeline.

[0003] Machine learning models are typically trained and evaluated in a model training environment. A human developer familiar with model development and training typically oversees model training and evaluation. Once the model performs as desired, it is deployed. Machine learning model deployment is the process of placing a trained machine learning model in a production environment where the model will be used for its intended purpose on live (non-training) data. Machine learning model deployment typically requires data from the model itself, data from the training environment in which the model was trained, and data from the production environment where the model will be deployed. Summary of the Invention

[0004] Exemplary embodiments provide methods, systems, and computer program products. Embodiments include a method using exported data of a machine learning model and a model training environment specification to identify a resource usage specification for the model and a code module usage specification for the model, where the resource usage specification specifies a set of resources used during training of the model in a model training environment specified by the model training environment specification, and the code module usage specification specifies a set of code modules used during training of the model in the model training environment. Embodiments include determining a code module installation specification from a code module requirements specification and a target execution environment specification, where the code module requirements specification specifies a set of code modules required to execute the model, and the code module installation specification specifies a set of code modules to be installed in a target execution environment, the target execution environment including an environment for execution of the model, the target execution environment being specified by the target execution environment specification. Embodiments cause installation of the code modules specified by the code module installation specification in the target execution environment, where the installation results in an updated target execution environment. Embodiments use the updated target execution environment data to validate the updated target execution environment for execution of the model. Embodiments simulate execution of the model in the updated target execution environment. Embodiments deploy the model in the updated target execution environment in response to the simulating being successful.

[0005] Embodiments include a computer-usable program product that includes one or more computer-readable storage devices and program instructions stored on at least one of the one or more storage devices.

[0006] An embodiment includes a computer system including one or more processors, one or more computer-readable memories, one or more computer-readable storage devices, and program instructions stored in at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories. [Brief explanation of the drawings]

[0007] The particular novel features which are believed characteristic of the invention are set forth in the appended claims. However, the invention itself, together with its preferred mode of use, further objects and advantages, will best be understood by reference to the following detailed description of illustrative embodiments when read in conjunction with the accompanying drawings.

[0008] [Figure 1] 1 depicts an exemplary diagram of a data processing environment in which illustrative embodiments may be implemented;

[0009] [Figure 2] FIG. 1 illustrates a block diagram of an exemplary configuration for automated machine learning model deployment according to an exemplary embodiment.

[0010] [Figure 3] 1 illustrates an example of automated machine learning model deployment according to an exemplary embodiment.

[0011] [Figure 4] 1 illustrates exemplary results of automated machine learning model deployment according to an exemplary embodiment.

[0012] [Figure 5] 1 illustrates exemplary results of automated machine learning model deployment according to an exemplary embodiment.

[0013] [Figure 6A] 1 illustrates a flowchart of an exemplary process for automated machine learning model deployment according to an exemplary embodiment.

[0014] [Figure 6B] 1 illustrates a continuous flowchart of an exemplary process for automated machine learning model deployment according to an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] The illustrative embodiments recognize that while tools for automatically configuring and training machine learning models are currently available, tools for automatically deploying machine learning models are not currently available. Furthermore, deployment is more complex than simply copying application files from one location to another and starting execution in the new location. Currently available automated model development tools typically build custom models by combining existing components. Often, the tools explore different model architectures before selecting the best-performing model. As a result, models have diverse architectures and use components, or code modules, from a diverse set of existing software libraries or software packages. For example, some common libraries in the Python computer language are sklearn, xgboost, tsfresh, and lifeline. (Python is a registered trademark of the Python Software Foundation in the United States and other countries.) Required software libraries must be installed in the environment in which the model is to run; otherwise, the model will not function properly. However, automated tools typically do not identify required software libraries, and dependencies between components can be difficult for human end users who are unfamiliar with deploying models and keeping track of the details of each individual model deployed. In addition, other resources, such as processor and memory, must be sufficient for the model and its software library to function at a specified speed and accuracy. For example, a model trained to monitor and adjust the operation of a wind turbine according to changing environmental conditions, but that reacts too slowly to incoming severe weather, may cause the turbine to be damaged by the severe weather. Some mission-critical applications require that a model have proof that it meets specific speed or response requirements in a specific execution environment. Therefore, different models are likely to have different, non-standard deployment environment requirements that are difficult for humans to evaluate.Similarly, determining why a model fails to deploy or operate correctly when operating in an environment that is incorrectly configured for the model can be difficult and time-consuming, especially for end users unfamiliar with the details of the model.

[0016] The illustrative embodiments recognize that one solution is to replicate the model's training environment in the production environment, since it is known that the training environment already has sufficient resources and accurate software libraries. However, because model training is typically more resource-intensive than simply using a pre-trained model, the training environment often includes resources and software libraries that are unnecessary in the deployment environment. For example, a model may be trained using 64 processors and 16 gigabytes of memory, but once trained, it may require only 4 processors and 4 gigabytes of memory. Allocating unused resources and installing unused libraries is inefficient. Allocating unused resources and installing unused libraries also adds unnecessary expense in an environment where these services are billed separately.

[0017] The illustrative embodiments also recognize that model users have a variety of platforms with different characteristics on which their models are deployed. One platform allows users to customize the model execution environment, while another platform offers more than 40 default environment configurations that can be further customized. Additionally, for speed, data security, or other reasons, some users prefer that the model and the data processed by the model run in the same environment, on the same physical system, or in the same data center. However, because different models have different deployment environment requirements, it can be difficult for a human end user unfamiliar with the details of the model to optimize a selected deployment environment for a particular model. The illustrative embodiments also recognize that while existing deployment tools typically test model accuracy, such tools do not test the speed or response time of the model or perform validation and testing of the model for execution in the selected deployment environment.

[0018] Thus, the illustrative embodiments recognize that an unmet need exists for an automated tool that determines the requirements of a model when run in production (as opposed to training), selects a suitable deployment environment, validates and tests the model for execution as specified in the deployment environment, and deploys all models in an automated and standardized manner.

[0019] The illustrative embodiments recognize that currently available tools or solutions do not address or provide adequate solutions to these needs. The illustrative embodiments used to describe the present invention generally address and solve the above-mentioned problems and other problems associated with automated machine learning model deployment.

[0020] Embodiments are implemented as a software application. An application implementing embodiments may be configured as a modification to an existing machine learning model management or execution system, as a separate application that operates in conjunction with an existing machine learning model management or execution system, as a standalone application, or some combination thereof.

[0021] In particular, some example embodiments provide a method for identifying resource usage and code module usage specifications of an exported model, determining code module installation specifications, causing installation of code modules specified by the code module installation specifications in a target execution environment, validating the updated target execution environment for execution of the trained model, simulating execution of the model in the updated target execution environment, and, in response to successful simulation, deploying the model in the updated target execution environment.

[0022] An embodiment receives exported data of a trained machine learning model to be deployed and an environment specification of the environment in which the model was trained. The trained machine learning model data includes all model data necessary to run the model in another execution environment. Some currently available automated model training tools (e.g., AutoAI, a currently available tool that automates steps in building predictive machine learning models) include model export capabilities that generate exported data. An embodiment also receives an environment specification of the environment in which the model was trained. The embodiment's specification specifies characteristics of the model training environment, such as the number of processors, the amount of memory, and the software modules installed in the environment.

[0023] Embodiments use exported data of a trained machine learning model to be deployed and an environment specification of the environment in which the model was trained to identify a resource usage specification. The resource usage specification specifies a set of resources for the model training environment. Some non-limiting examples of resources for the model training environment are the number and speed of processors, the amount of memory, and the number, architecture, and speed of any graphics processors used to supplement the processors with specialized computations. For example, one resource usage specification may specify that the model training environment has 64 processors and 16 gigabytes of memory.

[0024] Embodiments use exported data for a trained machine learning model to be deployed and an environment specification for the environment in which the model was trained to identify a code module usage specification for the model. The code usage specification specifies the set of code modules or libraries present in the model training environment, as well as the version number of each code module, if available. For example, one code usage specification might specify that the training environment includes version 1.0 of the scikit-learn Python library.

[0025] Because not every code module present in the model training environment is required to execute the model, an embodiment analyzes the exported data and prunes the code module usage specification to remove one or more code modules that are not actually required. The pruning results in a code module requirements specification that specifies the set of code modules required to execute the model in the execution environment. To prune the code module usage specification, an embodiment generates control flow data for the model and uses the control flow data to trace the use of specific code modules as models executed in the training environment. Only code modules actually used during model execution remain in the code module requirements specification. To generate control flow data for an exported model implemented in the Python language, one embodiment uses currently available Python tools (e.g., ast) to generate an abstract syntax tree, which is a record of the Python interpreter's interpretation of the Python program's source code, including the program's control flow data. Techniques for generating control flow data for exported models in other computer languages ​​are also currently available.

[0026] Embodiments use a model training environment specification to select a target execution environment. The target execution environment is the environment into which a model is intended to be deployed for execution on live (non-training) data. The target execution environment is specified by a target execution environment specification. The target execution environment need not have the same resources, code modules, or versions of code modules as the training execution environment. In embodiments, the target execution environment has fewer processors, less memory, or a different graphics processor architecture than the training execution environment. In embodiments, the target execution environment lacks code modules present in the training execution environment or has older versions of code modules than the training execution environment. In one embodiment, a user provides the target execution environment specification. For example, a wind turbine monitoring model is required to run in the same environment that collects the wind turbine data that the model is intended to process, and reconfiguring the data collection environment requires the addition or replacement of specific physical components. In another embodiment, a user provides preferences or constraints (e.g., specifying the provider of the target execution environment or that the target execution environment be located in a particular country or data center), and embodiments select a target execution environment that meets the preferences or constraints. Another embodiment selects a target execution environment without user input. To select a target execution environment, an embodiment analyzes data for one or more available execution environments and ranks the environments according to one or more of the time to install additional components, the additional memory required if existing packages cannot be removed, the frequency with which library versions change, and the cost of making the required resources available in the execution environment. The embodiment then selects the highest-ranked available environment.

[0027] Embodiments determine a code module installation specification from a target execution environment specification and a code module requirements specification. The code module installation specification specifies a set of code modules to be installed in the target execution environment. In particular, embodiments compare data of code modules currently installed in the target execution environment (specified by the target execution environment specification or obtained separately from the target environment specified by the target execution environment specification) with data of code modules required to be installed in the target execution environment (specified by the code module requirements specification). Embodiments add any code modules that are not already installed or that have been installed but are of an earlier version to the code module installation specification.

[0028] Embodiments install one or more code modules specified by a code module installation specification in a target execution environment. The installing updates the target execution environment. In one embodiment, the code module installation is performed only after determining that installing the code modules will not cause the target execution environment to exceed resource limitations specified by the target execution environment specification. For example, installing a large code module in an environment that is already nearly out of memory may cause the environment to actually run out of memory, which is undesirable. If embodiments determine that installing the code modules will cause the target execution environment to exceed resource limitations or if an error occurs during installation, embodiments notify the user and select a different target execution environment and repeat the code module installation specification and installation in that environment, or take other corrective action.

[0029] Embodiments also adjust resource usage specifications, code module installation specifications, or both to specify fewer resources or code modules than originally specified, or different versions of code modules. Typically, deployed models require fewer resources and code modules than the model to be trained, allowing for more efficient computing resource usage and saving money by adjusting the target environment requirements to more closely align with what the deployed model actually requires.

[0030] Embodiments use the resource usage specification and data from the updated target execution environment to validate the resources of the updated target execution environment for execution of the trained model. Embodiments also use the code module installation specification and data from the updated target execution environment to validate the code modules of the updated target execution environment for execution of the trained model. Validation checks that the target execution environment has sufficient resources and that the code modules are installed and have the correct versions, and then executes the trained model. Validation before model deployment avoids several sources of model deployment failure, and thus avoids the need to debug such failures, particularly in non-standard execution environments.

[0031] Embodiments simulate the execution of the model in the updated target execution environment. Simulating checks that the model executes correctly in the simulated version of the target execution environment. Simulating before model deployment avoids several sources of model deployment failures, and therefore avoids the need to debug such failures specifically in non-standard execution environments. To perform the simulation, embodiments use currently available technology.

[0032] If the simulation fails due to an error or does not meet the specified performance specifications, embodiments readjust the resource usage specifications, the code module installation specifications, or both to specify additional resources or code modules, or different versions of code modules, and repeat the verification and simulation using the readjusted specifications.

[0033] On the other hand, if the simulation is successful, embodiments deploy the model in the updated target execution environment. Once deployed, the model can run in the updated target execution environment and process live data. To perform the deployment, embodiments use currently available technology.

[0034] Embodiments maintain a record of the deployment, including exported data of the trained machine learning model to be deployed, environmental specifications of the environment in which the model was trained, the target execution environment, any adjustments made to the target execution environment, and the results of validation, simulation, and deployment, which can be used to adjust one or more of the methods described herein.

[0035] The automated machine learning model deployment approach described herein is not available in currently available methods within the technical endeavor related to machine learning model management. The method of the embodiments described herein, when implemented for execution on a device or data processing system, includes: a substantial advance in the device's or data processing system's capabilities in identifying resource usage and code module usage specifications for the exported model; determining a code module installation specification; causing installation of the code modules specified by the code module installation specification in a target execution environment; validating the updated target execution environment for execution of the trained model; simulating execution of the model in the updated target execution environment; and deploying the model in the updated target execution environment in response to the simulating step being successful.

[0036] The exemplary embodiments are described with respect to particular types of execution environments, resources, code modules, computer languages, specifications, models, verifications, calibrations, sensor measurements, devices, data processing systems, environments, components, and applications, by way of example only. Any particular manifestations of these and other similar artifacts are not intended to limit the invention. Any suitable manifestations of these and other similar artifacts may be selected within the scope of the exemplary embodiments.

[0037] Furthermore, exemplary embodiments may be implemented with respect to any type of data, data source, or access to a data source via a data network. Within the scope of the present invention, any type of data storage device may provide data to embodiments of the present invention, either locally at a data processing system or via a data network. Where embodiments are described using a mobile device, within the scope of exemplary embodiments, any type of data storage device suitable for use with a mobile device may provide data to such embodiments, either locally at the mobile device or via a data network.

[0038] The exemplary embodiments are described using specific code, designs, architectures, protocols, layouts, diagrams, and tools, by way of example only, and are not limited to the exemplary embodiments. Furthermore, the exemplary embodiments are described in some instances using specific software, tools, and data processing environments, by way of example only, for clarity of explanation. The exemplary embodiments may be used with other equivalent or similar purpose structures, systems, applications, or architectures. For example, other equivalent mobile devices, structures, systems, applications, or architectures therefor may be used with such embodiments of the present invention within the scope of the present invention. The exemplary embodiments may be implemented in hardware, software, or a combination thereof.

[0039] Examples in this disclosure are used for clarity of explanation only and are not intended to limit the exemplary embodiments. Additional data, operations, actions, tasks, activities, and operations are contemplated by this disclosure and are contemplated within the scope of the exemplary embodiments.

[0040] Any advantages listed herein are merely examples and are not intended to limit the exemplary embodiments. Additional or different advantages may be realized by particular exemplary embodiments. Furthermore, particular exemplary embodiments may have some, all, or none of the above-listed advantages.

[0041] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings cited herein is not limited to cloud computing environments. Rather, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0042] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that are rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0043] The characteristics are as follows:

[0044] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time and network storage, automatically as needed without requiring human interaction with the service provider.

[0045] Wide network access: Functionality is available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (eg, cell phones, laptops, and PDAs).

[0046] Resource Pooling: Provider computing resources are pooled to serve multiple consumers using a multi-tenant model, with various physical and virtual resources dynamically allocated and reallocated according to demand. There is location independence in that consumers generally have no control or knowledge over the exact location of the resources provided, although it may be possible to specify location at a higher level of abstraction (e.g., country, state, or data center).

[0047] Rapid Elasticity: Capacity is provisioned quickly and elastically, sometimes automatically, and can be quickly scaled out or quickly released and quickly scaled in. In many cases, the capacity available for provisioning appears unlimited to the consumer, and can be purchased in any quantity at any point in time.

[0048] Metering Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts) at a certain level of abstraction. Resource usage is monitored, controlled, reported, and billed, providing transparency to both providers and consumers of the services being used.

[0049] The service model is as follows:

[0050] Software as a Service (SaaS): The consumer is offered the ability to use a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0051] Platform as a Service (PaaS): The ability offered to consumers is to deploy applications they create or acquire, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does control the deployed applications and, in some cases, the configuration of the application hosting environment.

[0052] Infrastructure as a Service (IaaS): The ability provided to consumers is to provision processing, storage, network, and other basic computing resources onto which they can deploy and run any software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather control the operating systems, storage, deployed applications, and in some cases, limited control over selected networking components (e.g., host firewalls).

[0053] The deployment model is as follows:

[0054] Private Cloud: Cloud infrastructure is operated exclusively for an organization. It may be managed by the organization or a third party and may reside on-premise or off-premise.

[0055] Community Cloud: Cloud infrastructure is shared by multiple organizations to support a specific community with shared concerns (e.g., mission, security requirements, policies, and regulatory compliance considerations). Community clouds may be managed by those organizations or by a third party and may exist on-premises or off-premises.

[0056] Public Cloud: Cloud infrastructure is made available to the general public or large industry groups and is owned by organizations that sell cloud services.

[0057] Hybrid Cloud: This cloud infrastructure is a composite of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technologies that allow for data and application portability (e.g., cloud bursting for load balancing between clouds).

[0058] Cloud computing environments are service-oriented with an emphasis on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0059] Various aspects of the present disclosure are described through text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of a computer program product (CPP). For any flowchart, depending on the technology involved, operations may be performed in an order different from that shown in a given flowchart. For example, again depending on the technology involved, two operations shown in successive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in an at least partially overlapping manner.

[0060] A computer program product embodiment is a term used in this disclosure to describe any set of one or more storage media (also referred to as "mediums") collectively contained in one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing the computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. The computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices (such as pits / lands formed on a major surface of a punch card or disk), or any suitable combination of the foregoing. Computer-readable storage media, as the term is used in this disclosure, is not to be construed as storage in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through fiber optic cables, electrical signals communicated through wires, and / or other transmission media. As will be appreciated by those skilled in the art, data is typically moved at some infrequent time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but the above does not make a storage device temporary because the data is not temporary while it is stored.

[0061] Referring now to the figures, and in particular to FIG. 1, this figure is an exemplary diagram of a data processing environment in which illustrative embodiments may be implemented. FIG. 1 is merely an example and is not intended to assert or imply any limitation with respect to the environments in which different embodiments may be implemented. Particular implementations may make many modifications to the depicted environment based on the following description. FIG. 1 illustrates a block diagram of a network of data processing systems in which illustrative embodiments may be implemented. Computing environment 100 includes an example environment for the execution of at least some of the computer code associated with performing the inventive methodology, such as application 200. Application 200 implements an embodiment of the automated machine learning model deployment described herein. In addition to block 200, computing environment 100 may include, for example, computer 101, wide area network (WAN) 102, end user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes a set of processors 110 (including processing circuitry 120 and cache 121), a communications fabric 111, volatile memory 112, persistent storage 113 (including operating system 122 and block 200, as identified above), a set of peripheral devices 114 (including a user interface (UI), a set of devices 123, storage 124, and a set of Internet of Things (IoT) sensors 125), and a network module 115. Remote server 104 includes a remote database 130. Public cloud 105 includes a gateway 140, a cloud orchestration module 141, a set of host physical machines 142, a set of virtual machines 143, and a set of containers 144. Application 200, unless explicitly disambiguated, executes on computer 101, end-user device 103, remote server 104, or any of the computers in public cloud 105 or private cloud 106.The application 200 may deploy the model either on the computer 101, on the end-user device 103, on a remote server 104, on a public cloud 105 or a private cloud 106, or on a computer in another environment.

[0062] Computer 101 may take the form of a desktop computer, a laptop computer, a tablet computer, a smartphone, a smartwatch or other wearable computer, a mainframe computer, a quantum computer, or any other form of computer or mobile device now known or later developed that is capable of executing programs, accessing a network, or querying a database, such as remote database 130. As is well understood in the field of computer technology, and depending on the technology, execution of a computer-implemented method may be distributed among multiple computers and / or among multiple locations. However, in this description of computing environment 100, to keep the description as concise as possible, the detailed discussion focuses on a single computer, specifically computer 101. Although computer 101 is not shown in FIG. 1 within the cloud, it may be located within the cloud. However, computer 101 is not required to reside within the cloud except to any extent expressly indicated.

[0063] Processor set 110 includes one or more computer processors of any type now known or later developed. Processor set 110 may include one or more processors and may be implemented using one or more heterogeneous processor systems. The processors in processor set 110 may be single- or multi-core processors or graphics processors. Processing circuitry 120 may be distributed across multiple packages, e.g., multiple tailored integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for fast access by threads or cores executing on processor set 110. Cache memory is typically organized into multiple levels depending on relative proximity to the processing circuitry. Alternatively, some or all caches for a processor set may be located “off-chip.” In some computing environments, processor set 110 may be designed to operate with qubits and perform quantum computing.

[0064] Operating system 122 runs on computer 101. Operating system 122 cooperates to provide control of various components within computer 101. Instructions for operating system 122 are located in a storage device, such as persistent storage 113, and may be loaded into at least one of one or more memories, such as volatile memory 112, for execution by processor set 110.

[0065] Computer-readable program instructions are typically loaded onto computer 101 to cause processor set 110 of computer 101 to perform a series of operational steps, thereby realizing a computer-implemented method, such that the instructions so executed instantiate the method specified in the flowcharts and / or descriptions of the computer-implemented methods contained herein (collectively referred to as the "invention methods"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by processor set 110 to control and direct the execution of the inventive methods. In computing environment 100, at least some of the instructions for implementing the inventive methods of application 200 may be stored in persistent storage 113 and loaded into at least one of one or more memories, such as volatile memory 112, for execution by processor set 110. The processes of the exemplary embodiments are executed by the processor set 110 using computer-implemented instructions, which may be located in memory, such as volatile memory 112, persistent storage 113, or in one or more peripheral devices in the peripheral device set 114. Furthermore, in one case, the application 200 is downloaded from the remote server 104 over the WAN 102, where the same code is stored in a storage device. In another case, the application 200 is downloaded to the remote server 104 over the WAN 102, where the downloaded code is stored in a storage device.

[0066] Communications fabric 111 is the signal-conducting pathway that allows various components of computer 101 to communicate with one another. Typically, this fabric is made up of switches and conductive pathways, such as switches and conductive pathways that make up buses, bridges, physical input / output ports, etc. Other types of signal communication pathways may be used, such as fiber optic and / or wireless communication pathways.

[0067] Volatile memory 112 may be any type of volatile memory now known or later developed. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory is characterized by random access, although this is not required unless expressly indicated. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to computer 101.

[0068] Persistent storage 113 is any form of non-volatile storage for a computer, now known or later developed. The non-volatility of this storage means that stored data remains regardless of whether power is supplied to computer 101 and / or to persistent storage 113 directly. While persistent storage 113 can be read-only memory (ROM), typically at least a portion of persistent storage allows data to be written, data to be deleted, and data to be rewritten. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. Operating system 122 may take several forms, such as various known proprietary operating systems or open-source Portable Operating System Interface-type operating systems that utilize a kernel. The code included in application 200 typically includes at least some of the computer code associated with performing the methods of the invention.

[0069] The peripheral device set 114 includes the set of peripheral devices of the computer 101. Data communication connections between the peripheral devices and other components of the computer 101 may be implemented in various ways, such as Bluetooth® connections, near field communication (NFC) connections, connections formed by cables (such as universal serial bus (USB)-type cables), insertion-type connections (e.g., Secure Digital (SD) cards), connections formed through local area communication networks, and even connections formed through wide area networks such as the Internet. In various embodiments, the user interface (UI) device set 123 may include components such as a display screen, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. The storage 124 may be external storage, such as an external hard drive, or insertable storage, such as an SD card. The storage 124 may be persistent and / or volatile. In some embodiments, the storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 is required to have a large amount of storage (e.g., where computer 101 stores and manages large databases locally), this storage may be provided by a peripheral storage device designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple, geographically distributed computers. Internet of Things (IoT) sensor set 125 consists of sensors used in IoT applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0070] Network module 115 is a collection of computer software, hardware, and firmware that enables computer 101 to communicate with other computers over WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software-Defined Networking (SDN)), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for implementing the methods of the present invention may be downloaded to computer 101 from an external computer or external storage device, typically through a network adapter card or network interface included in network module 115.

[0071] Wide area network (WAN) 102 is any WAN (e.g., the Internet) capable of communicating computer data over non-local distances by any technology for communicating computer data now known or to be developed in the future. In some embodiments, a WAN may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include copper transmission cables, optical fiber transmissions, wireless transmissions, and computer hardware such as routers, firewalls, switches, gateway computers, and edge servers.

[0072] End-user device (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and may take any of the forms described above in connection with computer 101. EUD 103 typically receives useful and useful data from the operation of computer 101. For example, in the hypothetical case where computer 101 is designed to provide recommendations to the end user, the recommendations would typically be communicated from computer 101's network module 115 over WAN 102 to EUD 103. In this manner, EUD 103 can display or otherwise present the recommendations to the end user. In some embodiments, EUD 103 may be a client device such as a thin client, a heavy client, a mainframe computer, a desktop computer, and the like.

[0073] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 may be controlled and used by the same entity that operates computer 101. Remote server 104 represents a machine that collects and stores useful and useful data for use by other computers, such as computer 101. For example, in the hypothetical case where computer 101 is designed and programmed to provide recommendations based on historical data, this historical data may be provided to computer 101 from remote database 130 of remote server 104.

[0074] A public cloud 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer functionality, particularly data storage (cloud storage) and computing power, without direct active management by users. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct active management of the computing resources of the public cloud 105 is performed by computer hardware and / or software in a cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by virtual computing environments running on various computers that comprise a host physical machine set 142, which is the universe of physical computers within and / or available to the public cloud 105. The virtual computing environments (VCEs) typically take the form of virtual machines from a virtual machine set 143 and / or containers from a container set 144. It is understood that these VCEs are stored as images and transferred among and between various hosts of physical machines either as images or after instantiation of the VCE. Cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages active instantiations of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that enables public cloud 105 to communicate over WAN 102.

[0075] Some further description of virtualized computing environments (VCEs) is now provided. A VCE can be stored as an "image." A new, active instance of a VCE can be instantiated from the image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to a feature of an operating system in which the kernel allows the existence of multiple isolated user space instances, called containers. These isolated user space instances typically behave as actual computers from the perspective of programs running within them. A computer program running on a typical operating system can utilize all of the computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of the container and of the devices assigned to the container; this feature is known as containerization.

[0076] A private cloud 106 is similar to a public cloud 105, except that the computing resources are available only for use by a single enterprise. While the private cloud 106 is shown in communication with the WAN 102, in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. While each of the multiple clouds remains a separate, discrete entity, the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the constituent clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0077] 2, which illustrates a block diagram of an exemplary configuration for automated machine learning model deployment according to an exemplary embodiment. Application 200 is the same as application 200 in FIG.

[0078] Application 200 receives exported data of a trained machine learning model to be deployed and an environment specification of the environment in which the model was trained. The trained machine learning model data includes all model data necessary to execute the model in another execution environment. Application 200 also receives an environment specification of the environment in which the model was trained. In one embodiment, the specification specifies characteristics of the model training environment, such as the number of processors, the amount of memory, and the software modules installed in the environment.

[0079] The model characteristic identification module 210 uses the environment specification of the environment in which the model was trained to identify the exported data and resource usage specifications of the trained machine learning model to be deployed. The resource usage specification specifies a set of resources for the model training environment. Some non-limiting examples of resources for the model training environment are the number and speed of processors, the amount of memory, and the number, architecture, and speed of any graphics processors used to supplement the processors with specialized computations. For example, one resource usage specification might specify that the model training environment has 64 processors and 16 gigabytes of memory.

[0080] Module 210 also uses the exported data of the trained machine learning model to be deployed and the environment specification of the environment in which the model was trained to identify a code module usage specification for the model. The code usage specification specifies the set of code modules or libraries present in the model training environment, as well as the version number of each code module, if available. For example, one code usage specification might specify that the training environment includes version 1.0 of the scikit-learn Python library.

[0081] Because not every code module present in the model training environment is required to execute the model, code module pruning module 220 analyzes the exported data and prunes the code module usage specification to remove one or more code modules that are not actually required. The result of the pruning in the code module requirements specification specifies the set of code modules required to execute the model in the execution environment. To prune the code module usage specification, module 220 generates control flow data for the model and uses the control flow data to trace the use of specific code modules as models executed in the training environment. Only code modules actually used during model execution remain in the code module requirements specification. To generate control flow data for exported models implemented in the Python language, one implementation of module 220 uses currently available Python tools (e.g., ast) to generate an abstract syntax tree, which is a record of the Python interpreter's interpretation of the Python program's source code, including the program's control flow data.

[0082] The target execution environment selection module 230 uses the model training environment specification to select a target execution environment. The target execution environment is the environment into which the model is intended to be deployed for execution on live (non-training) data. The target execution environment is specified by the target execution environment specification. The target execution environment need not have the same resources, code modules, or versions of code modules as the training execution environment. In an implementation of module 230, the target execution environment has fewer processors, less memory, or a different graphics processor architecture than the training execution environment. In an implementation of module 230, the target execution environment lacks code modules present in the training execution environment or has older versions of code modules than the training execution environment. In one implementation of module 230, a user provides the target execution environment specification. For example, a wind turbine monitoring model is required to run in the same environment that collects the wind turbine data the model is intended to process, and reconfiguring the data collection environment requires the addition or replacement of certain physical components. In another implementation of module 230, a user provides preferences or constraints (e.g., specifying a provider of the target execution environment or that the target execution environment be located in a particular country or data center), and the implementation selects a target execution environment that meets the preferences or constraints. Another implementation of module 230 selects a target execution environment without user input. To select a target execution environment, module 230 analyzes data of one or more available execution environments and ranks the environments according to one or more of the time to install additional components, the additional memory required if existing packages cannot be removed, the frequency with which library versions change, and the cost of making the required resources available in the execution environment. Module 230 then selects the highest-ranked available environment.

[0083] Code module installation module 240 determines a code module installation specification from the target execution environment specification and the code module requirements specification. The code module installation specification specifies a set of code modules to be installed in the target execution environment. In particular, module 240 compares data of code modules currently installed in the target execution environment (specified by the target execution environment specification or obtained separately from the target environment specified by the target execution environment specification) with data of code modules required to be installed in the target execution environment (specified by the code module requirements specification). Module 240 adds any code modules (not already installed or installed but of an earlier version) to the code module installation specification.

[0084] Code module installation module 240 also installs one or more code modules specified by the code module installation specification in the target execution environment. The installing updates the target execution environment. In one implementation of module 240, code module installation is performed only after determining that installing the code module will not cause the target execution environment to exceed resource limitations specified by the target execution environment specification. For example, installing a large code module in an environment that is already nearly out of memory may actually cause the environment to run out of memory, which is undesirable. If module 240 determines that installing the code module will cause the target execution environment to exceed resource limitations or if an error occurs during installation, module 240 notifies the user, selects a different target execution environment, and repeats determining the code module installation specification and installing in that environment, or takes other corrective action.

[0085] Application 200 may also adjust resource usage specifications, code module installation specifications, or both to specify fewer resources or code modules than originally specified, or different versions of code modules. Typically, deployed models require fewer resources and code modules than the models to be trained, allowing for more efficient computing resource usage and saving money by adjusting the target environment requirements to more closely align with what the deployed model actually requires.

[0086] Verification module 250 uses the resource usage specification and the data of the updated target execution environment to verify the resources of the updated target execution environment for execution of the trained model. Module 250 also uses the code module installation specification and the data of the updated target execution environment to verify the code modules of the updated target execution environment for execution of the trained model. Validation checks that the target execution environment has sufficient resources and that the code modules are installed and have the correct versions before executing the trained model. Validation before model deployment avoids several sources of model deployment failure, and therefore avoids the need to debug such failures, particularly in non-standard execution environments.

[0087] The simulation module 260 simulates the execution of the model in the updated target execution environment. Simulating checks that the model executes correctly in the simulated version of the target execution environment. Simulating before model deployment avoids some sources of model deployment failures and therefore avoids the need to debug such failures specifically in non-standard execution environments.

[0088] If the simulation fails due to an error or does not meet the specified performance specifications, the application 200 readjusts the resource usage specifications, the code module installation specifications, or both to specify additional resources or code modules, or different versions of code modules, and repeats the verification and simulation using the readjusted specifications.

[0089] On the other hand, if the simulation is successful, the deployment module 270 deploys the model in the updated target execution environment. Once deployed, the model can run in the updated target execution environment and process live data.

[0090] Referring to Figure 3, this figure illustrates an example of automated machine learning model deployment according to an exemplary embodiment. The example may be performed using application 200 in Figure 1. Model characteristic identification module 210, code module pruning module 220, target execution environment selection module 230, code module installation module 240, verification module 250, simulation module 260, and deployment module 270 are the same as model characteristic identification module 210, code module pruning module 220, target execution environment selection module 230, code module installation module 240, verification module 250, simulation module 260, and deployment module 270 in Figure 2.

[0091] As shown, model characteristic identification module 210 uses machine learning model data 310 (exported data of the trained machine learning model to be deployed) and model training environment specification 312 (specifying the environment in which the model was trained) to identify resource usage specification 320. Module 210 also uses machine learning model data 310 and model training environment specification 312 to identify code module usage specification 330.

[0092] The code module pruning module 220 prunes the code module usage specification 330 to remove one or more code modules that are not actually required, resulting in a code module requirements specification 340 that specifies the set of code modules required to execute the model in the execution environment.

[0093] The target execution environment selection module 230 uses the model training environment specification 312 and the execution environment data 314 to select a target execution environment 350 . Code module installation module 240 determines a code module installation specification from target execution environment specification 350 and code module requirements specification 340. Code module installation module 240 also causes one or more code modules specified by the code module installation specification to be installed in the target execution environment, resulting in an updated target execution environment 360.

[0094] The validation module 250 validates the resources of the updated target execution environment for execution of the trained model using the resource usage specification 330 and the data of the updated target execution environment 360. The module 250 also validates the code modules of the updated target execution environment for execution of the trained model using the code module installation specification and the data of the updated target execution environment 360. The result is a validation status 370.

[0095] If the verification status 370 is successful, the simulation module 260 simulates the execution of the model in the updated target execution environment. The result is a simulation status 380. If the simulation status 380 is successful, the deployment module 270 deploys the model in the updated target execution environment (390).

[0096] Referring to FIG. 4, this figure illustrates example results of automated machine learning model deployment according to an example embodiment.

[0097] In particular, example 400 shows a table in which each column represents one scenario for deploying a machine learning pipeline or model using a different execution environment configuration. The left-most column holds a number corresponding to each different scenario, and the Algorithm column (second from the left) holds the algorithm used in the machine learning model. A pipeline may include components from multiple libraries, and the Installed Packages column (third from the left) indicates how many new libraries were required to be installed in the target execution environment. In some scenarios, the new libraries were not installed (e.g., Scenario 2), and in some cases (e.g., Scenario 7), an exception was raised due to a library installation error. The New Deployment Time column (second from the right) reports the time taken to prepare the target execution environment and make it available for use by the model. The right-most column indicates the time taken using currently available technology, thus illustrating the benefit of reduced deployment time using embodiments of the present invention.

[0098] Referring to FIG. 5, this figure illustrates example results of automated machine learning model deployment according to an example embodiment.

[0099] In particular, example 500 shows a table where each column represents one scenario of deploying a machine learning pipeline or model using a different execution environment configuration. The leftmost column holds a number corresponding to each different scenario, which are the same as those shown in example 400 in FIG. 4. Because different machine learning models have different dependencies, the Package column names the model package, the Package Size column reports the package size of the deployed model, and the Install Size column reports the total memory used if the model is deployed.

[0100] 6A, which illustrates a flowchart of an example process for automated machine learning model deployment according to an example embodiment. Process 600 may be implemented in application 200 in FIG.

[0101] At block 602, the application uses the exported data of the machine learning model and the model training environment specification to identify resource usage specifications for the model and code module usage specifications for the model. At block 604, the application prunes the code module usage specifications by analyzing the exported data, and the pruning results in a code module requirement specification. At block 606, the application uses the model training environment specification to select a target execution environment. At block 608, the application determines a code module installation specification from the target execution environment specification and the code module requirement specification. At block 610, the application checks whether installing the code module specified by the code module installation specification would exceed the resource limitations specified by the target execution environment specification. If yes at block 612 (the "Yes" path of block 610), the application selects another target execution environment and returns to block 608. Otherwise (the "No" path of block 610), at block 614, the application installs the code module specified by the code module installation specification in the target execution environment. The application then proceeds to point A.

[0102] Referring to Figure 6B, this figure shows a continued flowchart of an exemplary process of automated machine learning model deployment according to an exemplary embodiment. In particular, Figure 6B shows a continuation of process 600 in Figure 6A. Point A is the same as point A in Figure 6A.

[0103] From point A, in block 616, the application verifies the resources of the updated target execution environment and the code modules installed in the updated target execution environment for execution of the trained model. In block 618, the application checks whether the verification was successful. If yes (the "Yes" path of block 618), in block 620, the application simulates execution of the model in the updated target execution environment. In block 622, the application checks whether the simulation was successful. If yes (the "Yes" path of block 622), in block 624, the application deploys the model in the updated target execution environment. Then (also the "No" paths of blocks 618 and 622), the application terminates.

[0104] Accordingly, computer-implemented methods, systems or apparatus, and computer program products are provided in exemplary embodiments for automated machine learning model deployment and other related features, functions, or operations. When an embodiment, or a portion thereof, is described with respect to a certain type of device, computer-implemented method, system, or apparatus, the computer program product, or a portion thereof, is adapted or configured for use with a suitable and equivalent article for that type of device.

[0105] Where embodiments are described as being implemented in an application, provision of the application in a software-as-a-service (SaaS) model is contemplated within the scope of exemplary embodiments. In a SaaS model, the capabilities of an application implementing an embodiment are provided to users by running the application in a cloud infrastructure. Users can access the application through a thin-client interface, such as a web browser (e.g., web-based email) or other lightweight client application, using a variety of client devices. Users do not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage of the cloud infrastructure. In some cases, users may not even manage or control the capabilities of the SaaS application. In some other cases, a SaaS implementation of an application may allow for possible exceptions to limited user-specific application configuration settings.

[0106] The present invention may be a system, method and / or computer program product integrated at any possible level of technical detail. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0107] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may comprise copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions to a computer-readable storage medium within the respective computing / processing device for storage.

[0108] The computer-readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or either source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, C++, or the like, and procedural programming languages ​​such as the “C” programming language or the like. The computer-readable program instructions may run entirely on the user's computer, as a standalone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer readable program instructions to personalize the electronic circuitry by utilizing state information of the computer readable program instructions to perform aspects of the present invention.

[0109] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0110] These computer-readable program instructions are provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer-readable storage medium having instructions stored therein has an article of manufacture including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0111] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be executed on the computer, other programmable apparatus, or other device to generate a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0112] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may actually be realized as a single step, executed concurrently, substantially concurrently, partially, or fully in a time-overlapping manner, or the blocks may possibly be executed in reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks and / or flowchart diagrams in the block diagrams, are implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

Claims

1. using the exported data of the machine learning model and the model training environment specification to identify a resource usage specification for the model and a code module usage specification for the model, the resource usage specification specifying a set of resources to be used during training of the model in a model training environment specified by the model training environment specification, and the code module usage specification specifying a set of code modules to be used during training of the model in the model training environment; determining a code module installation specification from a code module requirements specification and a target execution environment specification, the code module requirements specification specifying a set of code modules required to execute the model, the code module installation specification specifying a set of code modules to be installed in a target execution environment, the target execution environment including an environment for execution of the model, the target execution environment being specified by the target execution environment specification; causing installation of the code modules specified by the code module installation specification in the target execution environment, the installing resulting in an updated target execution environment; validating the updated target execution environment for execution of the model using the updated target execution environment data; and simulating the execution of the model in the updated target execution environment; and deploying the model in the updated target execution environment in response to the simulating step being successful. A computer-implemented method comprising:

2. The computer-implemented method of claim 1 , wherein the model is trained in a training execution environment.

3. The computer-implemented method of claim 2 , wherein the training execution environment includes at least one resource that is not included in the target execution environment.

4. The computer-implemented method of claim 2 , wherein the training execution environment includes at least one code module that is not included in the target execution environment.

5. pruning the code module usage specification by analyzing the exported data, the pruning resulting in the code module requirements specification. The computer-implemented method of claim 1 further comprising:

6. 2. The computer-implemented method of claim 1, wherein the installing is performed in response to determining that installing the code modules specified by the code module installation specification will not exceed resource limitations specified by the target execution environment specification.

7. Validating the updated target execution environment for execution of the model using data of the updated target execution environment includes: validating resources of the updated target execution environment for execution of the model using the resource usage specification of the model. The computer-implemented method of claim 1 , comprising:

8. Validating the updated target execution environment for execution of the model using data of the updated target execution environment includes: using the code module installation specification to validate code modules installed in the updated target execution environment for execution of the model. The computer-implemented method of claim 1 , comprising:

9. 1. A computer program product comprising one or more computer-readable storage media and program instructions collectively stored on the one or more computer-readable storage media, the program instructions comprising: using the exported data of the machine learning model and the model training environment specification to identify a resource usage specification for the model and a code module usage specification for the model, the resource usage specification specifying a set of resources to be used during training of the model in a model training environment specified by the model training environment specification, and the code module usage specification specifying a set of code modules to be used during training of the model in the model training environment; determining a code module installation specification from a code module requirements specification and a target execution environment specification, the code module requirements specification specifying a set of code modules required to execute the model, the code module installation specification specifying a set of code modules to be installed in a target execution environment, the target execution environment including an environment for execution of the model, the target execution environment being specified by the target execution environment specification; causing installation of the code modules specified by the code module installation specification in the target execution environment, the installing resulting in an updated target execution environment; validating the updated target execution environment for execution of the model using data from the updated target execution environment; simulating execution of the model in the updated target execution environment; deploying the model in the updated target execution environment in response to the simulating step being successful.

1. A computer program product executable by a processor to cause the processor to perform operations comprising:

10. 10. The computer program product of claim 9, wherein the stored program instructions are stored on a computer-readable storage device within a data processing system, and the stored program instructions are transferred over a network from a remote data processing system.

11. The stored program instructions are stored in a computer readable storage device within a server data processing system, and the stored program instructions are downloaded in response to a request over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system, the remote data processing system further comprising: program instructions for metering usage of the program instructions associated with the request; and program instructions for generating a bill based on the metered usage 10. The computer program product of claim 9, comprising:

12. The computer program product of claim 9 , wherein the model is trained in a training execution environment.

13. The computer program product of claim 12 , wherein the training execution environment includes at least one resource that is not included in the target execution environment.

14. 13. The computer program product of claim 12, wherein the training execution environment includes at least one code module that is not included in the target execution environment.

15. pruning the code module usage specification by analyzing the exported data, the pruning resulting in the code module requirements specification.

10. The computer program product of claim 9, further comprising:

16. 10. The computer program product of claim 9, wherein the installing is performed in response to determining that installing the code modules specified by the code module installation specification would not exceed resource limitations specified by the target execution environment specification.

17. Validating the updated target execution environment for execution of the model using data of the updated target execution environment includes: validating resources of the updated target execution environment for execution of the model using the resource usage specification of the model.

10. The computer program product of claim 9, comprising:

18. Validating the updated target execution environment for execution of the model using data of the updated target execution environment includes: using the code module installation specification to validate code modules installed in the updated target execution environment for execution of the model.

10. The computer program product of claim 9, comprising:

19. 1. A computer system comprising a processor and one or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions comprising: using the exported data of the machine learning model and the model training environment specification to identify a resource usage specification for the model and a code module usage specification for the model, the resource usage specification specifying a set of resources to be used during training of the model in a model training environment specified by the model training environment specification, and the code module usage specification specifying a set of code modules to be used during training of the model in the model training environment; determining a code module installation specification from a code module requirements specification and a target execution environment specification, the code module requirements specification specifying a set of code modules required to execute the model, the code module installation specification specifying a set of code modules to be installed in a target execution environment, the target execution environment including an environment for execution of the model, the target execution environment being specified by the target execution environment specification; causing installation of the code modules specified by the code module installation specification in the target execution environment, the installing resulting in an updated target execution environment; validating the updated target execution environment for execution of the model using the updated target execution environment data; and simulating the execution of the model in the updated target execution environment; and deploying the model in the updated target execution environment in response to the simulating step being successful.

10. A computer system executable by a processor to cause the processor to perform operations comprising:

20. 20. The computer system of claim 19, wherein the model is trained in a training execution environment.

Images