Intelligent identification of execution environment

Abstract

A mechanism is provided for intelligently identifying an execution environment for executing a computing job. An execution time of the computing job in each of the plurality of execution environments is predicted by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environments. The predicted execution times of the machine learning models are aggregated. An aggregated predicted execution time of the computing job for the plurality of execution environments is compiled. In response to a selection of the execution environment from the plurality of execution environments based on the compilation of the aggregated predicted execution times of the computing job, the computing job is executed in the selected execution environment. Relevant data is collected during the execution of the computing job in the selected execution environment.

Description

[Technical Field]

[0001] This application relates generally to improved data processing apparatus and methods, and more particularly to mechanisms for intelligently identifying an execution environment for the execution of a computing job. [Background technology]

[0002] In computing, a job is a unit of work or a unit of execution (that performs that work). The components of a job (as a unit of work) are called tasks or steps (if sequential, such as a job stream). As a unit of execution, a job may be identified concretely by a single process, a process may have subprocesses (child processes, each corresponding to a job in the parent process) that perform the tasks or steps that make up the work of the job, or a job may be identified by a process group, or a job may be identified by an abstract reference to a process or process group.

[0003] Jobs may be started interactively, such as from the command line, or scheduled for non-interactive execution by a job scheduler, and then controlled by automatic or manual job control. Jobs with finite input may complete, successfully or unsuccessfully, or may fail to complete and eventually be terminated. In contrast, online processes, such as servers, have endless input (as long as the server is running, it will service requests), so they never complete and only stop when they are terminated (sometimes called "canceled").

[0004] The execution environment of a data processing system where a computing job is executed typically has several nodes in its execution flow, each performing a specific operation on input data and generating output data that can be consumed by downstream nodes or presented as a final result. Currently, users identify the execution flow of a computing job based on the nodes required to complete the execution of the computing job, and then select an execution environment for the computing job based on their experience and expectations to execute the computing job. However, if a user lacks experience or the execution flow of a computing job is complex, it can be difficult to correctly select the execution environment for the computing job. For example, if the execution flow of a computing job is lightweight, selecting a large-scale execution environment can result in resource waste and increased costs. As another example, if the execution flow of a computing job is heavy and the execution environment selected is small, the execution time of the computing job can be excessively long. Summary of the Invention

[0005] This Summary is provided to introduce in a simplified form a selection of concepts that are further described herein in the Detailed Description. This Summary is not intended to identify key elements or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0006] One exemplary embodiment provides a method for intelligently identifying an execution environment for executing a computing job in a data processing system. For each execution environment among a plurality of execution environments, the exemplary embodiment predicts the execution time of the computing job in the execution environment by applying a set of existing machine learning models that match the execution context information and key parameters of the computing job and the execution environment information of the execution environment, and forms an aggregated predicted execution time for the computing job in the execution environment by aggregating the predicted execution times of the machine learning models. The exemplary embodiment compiles the aggregated predicted execution times of the computing job for the plurality of execution environments. In response to selecting an execution environment from the plurality of execution environments based on the compilation of the aggregated predicted execution times of the computing job, the exemplary embodiment executes the computing job in the selected execution environment. The exemplary embodiment collects relevant data during execution of the computing job in the selected execution environment. In this manner, by providing an estimated execution time for the computing job in all available candidate execution environments, the exemplary embodiment provides a method for performance-based selection of an appropriate execution environment without wasting resources or additional costs.

[0007] In an exemplary embodiment, the selection of an execution environment from multiple execution environments is based on a compilation of the aggregated predicted execution time of the computing job, either by user selection or automatic selection based on predefined user settings. This provides the advantages of a user-controlled or fully automated system. Furthermore, in an exemplary embodiment, the execution context information is selected from a group including input data size, number of fields, number of categorical fields, number of continuous fields, univariate statistics of categorical fields, univariate statistics of continuous fields, metadata, and parameters, where the key parameters are identified based on values ​​associated with the execution context information for the computing job, and the execution environment information is selected from a group including number of processors, processor characteristics, memory size, software development information, and operating system information. Providing broad flexibility in the component for predicting the execution time of a computing job provides user flexibility in identifying an appropriate execution environment.

[0008] In an exemplary embodiment, the set of existing machine learning models is generated by: collecting execution context information for a plurality of test cases; identifying key parameters using values ​​associated with each of the execution context information for the plurality of test cases; collecting execution environment information for each of a plurality of execution environments; executing each test case in the plurality of test cases in each execution environment among the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generating a set of machine learning models based on a relationship between the execution time of each test case in the associated execution environment and the execution context of the associated test case; and assigning an initial weight to each machine learning model in the set of machine learning models. Further, in an exemplary embodiment, the set of machine learning models for each execution environment among the plurality of execution environments includes one base machine learning model that uses all of the execution context information of the test cases, the values ​​associated with each of the execution context information of the test cases, and all of the associated execution environment information, and at least one lite machine learning model that includes some but not all of the execution context information of the test cases, the values ​​associated with each of the execution context information of the test cases, and all of the associated execution environment information. Creating machine learning models in this way improves the accuracy of predicting the execution time of computing jobs when used in real time.

[0009] In an exemplary embodiment, updating the set of machine learning models utilized in identifying an execution environment for executing a computing job includes adding relevant data collected about the selected execution environment to a data cache; calculating a model accuracy based on the cached data in response to the number of records in the data cache being greater than a first predetermined value; retraining at least one light machine learning model in the set of machine learning models by adjusting weights of the light machine learning model based on the calculated model accuracy; and adjusting a weight of the light machine learning model in response to the number of records in the data cache being greater than a second predetermined value. acc ) and determine whether the accuracy of the base model is below a predefined threshold (T acc ), generating a new base machine learning model using the cached data and replacing an existing base machine learning model for the associated execution environment with the newly generated base machine learning model for the associated execution environment in the set of machine learning models. In an exemplary embodiment, each of the newly generated base machine learning models and lite machine learning models associated with the execution environment is retrained by assigning new weights. Retraining the machine learning models in this manner makes predictions of the execution time of computing jobs more accurate over time.

[0010] Another exemplary embodiment provides a computer program product including a computer-usable or readable medium having a computer-readable program thereon. When executed on a computing device, the computer-readable program causes the computing device to, in this exemplary embodiment, predict, for each execution environment among a plurality of execution environments, an execution time of a computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment, and form an aggregated predicted execution time of the computing job in the execution environment by aggregating the predicted execution times of the machine learning models. In this exemplary embodiment, the aggregated predicted execution times of the computing job for the plurality of execution environments are compiled. In response to selecting an execution environment from the plurality of execution environments based on the compilation of the aggregated predicted execution times of the computing job, the exemplary embodiment executes the computing job in the selected execution environment. The exemplary embodiment collects relevant data during execution of the computing job in the selected execution environment. In this manner, by providing an estimated execution time of the computing job in all available candidate execution environments, the exemplary embodiment provides a computer program product for performance-based selection of an appropriate execution environment without wasting resources or additional costs.

[0011] In this exemplary embodiment, the selection of an execution environment from multiple execution environments is based on a compilation of the aggregated predicted execution time of the computing job, either by user selection or automatic selection based on predefined user settings. This provides the advantages of a user-controlled or fully automated system. Furthermore, in this exemplary embodiment, the execution context information is selected from a group including input data size, number of fields, number of categorical fields, number of continuous fields, univariate statistics of categorical fields, univariate statistics of continuous fields, metadata, and parameters, where the key parameters are identified based on values ​​associated with the execution context information for the computing job, and the execution environment information is selected from a group including number of processors, processor characteristics, memory size, software development information, and operating system information. Providing broad flexibility in the component for predicting the execution time of a computing job provides user flexibility in identifying an appropriate execution environment.

[0012] In this exemplary embodiment, the set of existing machine learning models is generated by a computer-readable program further causing a computing device to: collect execution context information for a plurality of test cases; identify key parameters using values ​​associated with each of the execution context information for the plurality of test cases; collect execution environment information for each of a plurality of execution environments; execute each test case in the plurality of test cases in each execution environment among the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generate a set of machine learning models based on a relationship between the execution time of each test case in the associated execution environment and the execution context of the associated test case; and assign an initial weight to each machine learning model in the set of machine learning models. Further, in this exemplary embodiment, the set of machine learning models for each execution environment among the plurality of execution environments includes one base machine learning model that uses all of the execution context information of the test cases, the values ​​associated with each of the execution context information of the test cases, and all related execution environment information, and at least one lite machine learning model that includes some but not all of the execution context information of the test cases, the values ​​associated with each of the execution context information of the test cases, and all related execution environment information. Creating machine learning models in this way improves the accuracy of predicting the execution time of computing jobs when used in real time.

[0013] In an exemplary embodiment, a computer-readable program causes a computing device to update a set of machine learning models utilized in identifying an execution environment for executing a computing job by adding relevant data collected about the selected execution environment to a data cache; calculating a model accuracy based on the cached data in response to the number of records in the data cache being greater than a first predetermined value; retraining at least one light machine learning model in the set of machine learning models by adjusting weights of the light machine learning model based on the calculated model accuracy; and adjusting a weight of the light machine learning model in response to the number of records in the data cache being greater than a second predetermined value. acc ) and determine whether the accuracy of the base model is below a predefined threshold (T acc ), by generating a new base machine learning model using the cached data and replacing the existing base machine learning model for the associated execution environment in the set of machine learning models with the newly generated base machine learning model for the associated execution environment. In an exemplary embodiment, each of the newly generated base machine learning models and lite machine learning models associated with the execution environment is retrained by assigning new weights. Retraining the machine learning models in this manner makes predictions of the execution time of computing jobs more accurate over time.

[0014] In yet another exemplary embodiment, a system / apparatus is provided. The system / apparatus may include one or more processors and a memory coupled to the one or more processors. The memory may include instructions that, when executed by the one or more processors, cause the one or more processors to: predict, for each execution environment among a plurality of execution environments, an execution time of a computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; and form an aggregated predicted execution time of the computing job in the execution environment by aggregating the predicted execution times of the machine learning models. In this exemplary embodiment, the aggregated predicted execution times of the computing job for the plurality of execution environments are compiled. In response to selecting an execution environment from the plurality of execution environments based on the compilation of the aggregated predicted execution times of the computing job, the exemplary embodiment executes the computing job in the selected execution environment. In this exemplary embodiment, relevant data is collected during execution of the computing job in the selected execution environment. Thus, by providing estimated execution times for a computing job in all available candidate execution environments, this exemplary embodiment provides a system / apparatus for performance-based selection of an appropriate execution environment without wasting resources or additional costs.

[0015] In this exemplary embodiment, the selection of an execution environment from multiple execution environments is based on a compilation of the aggregated predicted execution time of the computing job, either by user selection or automatic selection based on predefined user settings. This provides the advantages of a user-controlled or fully automated system. Furthermore, in this exemplary embodiment, the execution context information is selected from a group including input data size, number of fields, number of categorical fields, number of continuous fields, univariate statistics of categorical fields, univariate statistics of continuous fields, metadata, and parameters, where the key parameters are identified based on values ​​associated with the execution context information for the computing job, and the execution environment information is selected from a group including number of processors, processor characteristics, memory size, software development information, and operating system information. Providing broad flexibility in the component for predicting the execution time of a computing job provides user flexibility in identifying an appropriate execution environment.

[0016] In this exemplary embodiment, the set of existing machine learning models is generated by the instructions further causing the processor to: collect execution context information for a plurality of test cases; identify key parameters using values ​​associated with each of the execution context information for the plurality of test cases; collect execution environment information for each of a plurality of execution environments; execute each test case in the plurality of test cases in each execution environment among the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generate a set of machine learning models based on a relationship between the execution time of each test case in the associated execution environment and the execution context of the associated test case; and assign an initial weight to each machine learning model in the set of machine learning models. Further, in this exemplary embodiment, the set of machine learning models for each execution environment among the plurality of execution environments includes one base machine learning model that uses all of the execution context information of the test cases, the values ​​associated with each of the execution context information of the test cases, and all related execution environment information, and at least one lite machine learning model that includes some but not all of the execution context information of the test cases, the values ​​associated with each of the execution context information of the test cases, and all related execution environment information. Creating machine learning models in this way improves the accuracy of predicting the execution time of computing jobs when used in real time.

[0017] In this exemplary embodiment, the instructions cause the processor to update a set of machine learning models utilized in identifying an execution environment for executing a computing job by adding relevant data collected about the selected execution environment to a data cache; calculating a model accuracy based on the cached data in response to the number of records in the data cache being greater than a first predetermined value; retraining at least one light machine learning model in the set of machine learning models by adjusting weights of the light machine learning model based on the calculated model accuracy; and adjusting a weight of the light machine learning model in response to the number of records in the data cache being greater than a second predetermined value. acc ) and determine whether the accuracy of the base model is below a predefined threshold (T acc ), by generating a new base machine learning model using the cached data and replacing the existing base machine learning model for the associated execution environment in the set of machine learning models with the newly generated base machine learning model for the associated execution environment. In an exemplary embodiment, each of the newly generated base machine learning models and lite machine learning models associated with the execution environment is retrained by assigning new weights. Retraining the machine learning models in this manner makes predictions of the execution time of computing jobs more accurate over time.

[0018] Yet another exemplary embodiment provides a method for intelligently identifying an execution environment for executing a computing job in a data processing system. The exemplary embodiment determines an aggregated predicted execution time of the computing job in each execution environment of a plurality of execution environments. The exemplary embodiment executes the computing job in the selected execution environment in response to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job. The exemplary embodiment collects relevant data during execution of the computing job in the selected execution environment. In this manner, by providing an estimated execution time of the computing job in all available candidate execution environments, the exemplary embodiment provides a method for performance-based selection of an appropriate execution environment without wasting resources or additional costs.

[0019] Another exemplary embodiment provides a computer program product including a computer-usable or readable medium having a computer-readable program thereon. The computer-readable program, when executed on a computing device, causes the computing device to, in this exemplary embodiment, determine an aggregated predicted execution time of a computing job in each of a plurality of execution environments. In this exemplary embodiment, the computing job is executed in the selected execution environment in response to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job. In this exemplary embodiment, relevant data is collected during execution of the computing job in the selected execution environment. In this manner, by providing an estimated execution time of the computing job in all available candidate execution environments, the exemplary embodiment provides a computer program product for performance-based selection of an appropriate execution environment without wasting resources or additional costs.

[0020] These and other features and advantages of the present invention are described in, or will become apparent to, those skilled in the art upon consideration of, the following detailed description of exemplary embodiments of the invention.

[0021] The present invention, as well as its preferred modes of use and further objects and advantages, will best be understood by reference to the following detailed description of illustrative embodiments when read in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is an exemplary diagram of a distributed data processing system in which aspects of the illustrative embodiments may be implemented; [Figure 2] FIG. 1 is an exemplary block diagram of a computing device in which aspects of the exemplary embodiments may be implemented. [Figure 3] FIG. 1 illustrates an example of a functional block diagram of a data processing system for intelligently identifying an execution environment for the execution of a computing job, according to one illustrative embodiment. [Figure 4] FIG. 10 illustrates an example flow diagram of operations performed by an intelligent execution environment identification mechanism in identifying an execution environment for the execution of a new computing job, according to one exemplary embodiment. [Figure 5] FIG. 1 illustrates an example flow diagram of operations performed in generating a set of existing machine learning models for use by a runtime estimation engine of an intelligent execution environment identification mechanism in predicting the execution time of an execution environment, according to one illustrative embodiment. [Figure 6] FIG. 10 illustrates an example flow diagram of operations performed by a machine learning model update engine in updating a machine learning model utilized by an execution environment selection engine in identifying an execution environment for the execution of a new computing job, according to one exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0023] As mentioned above, the execution environment of a data processing system typically has several nodes in its execution flow, each performing a specific operation on input data and generating output data that can be consumed by downstream nodes or presented as a final result. Currently, users identify the execution flow of a computing job based on the nodes required to complete the execution of the computing job, and then select an execution environment for the computing job based on their experience and expectations to execute the computing job. However, if a user lacks experience or the execution flow of a computing job is complex, it can be difficult to correctly select the execution environment for the computing job. For example, if the execution flow of a computing job is lightweight, selecting a large-scale execution environment can result in resource waste and increased costs. As another example, if the execution flow of a computing job is heavy and the execution environment selected is small, the execution time of the computing job can be excessively long.

[0024] Thus, an exemplary embodiment provides a mechanism for intelligently identifying an execution environment for executing a computing job. The mechanism constructs a machine learning model to reflect the relationship between the job's execution time and execution context, e.g., the execution environment, input data information, key parameters, etc. Then, prior to the actual execution of the computing job, the mechanism estimates the execution time for executing the execution flow of the computing job in each candidate execution environment among the plurality of execution environments based on the machine learning model of each execution environment among the plurality of execution environments, to provide several combinations of execution environment and execution time as options for selection. That is, the mechanism may notify a user of the job's scale (heavy or light) and the estimated execution time in all available candidate execution environments, allowing the user to select an appropriate execution environment based on performance, or the mechanism may automatically select an execution environment based on predefined user settings. After the execution of each computing job in the selected execution environment, the mechanism collects execution information and refreshes the associated machine learning model.

[0025] Before beginning a discussion of various aspects of the exemplary embodiments and the improved computer operations performed by the exemplary embodiments, it should first be understood that throughout this description, the term "mechanism" will be used to refer to elements of the present invention that perform various operations, functions, etc. A "mechanism," as that term is used herein, may be an implementation of a function or aspect of the exemplary embodiments in the form of an apparatus, a procedure, or a computer program product. In the case of a procedure, the procedure is implemented by one or more devices, apparatuses, computers, data processing systems, etc. In the case of a computer program product, the logic represented by computer code or instructions embodied in or on the computer program product is executed by one or more hardware devices to implement the function or perform the operations associated with the particular "mechanism." Thus, the mechanisms described herein may be implemented as dedicated hardware, software that executes on the hardware to configure the hardware to implement specific functions of the present invention that the hardware cannot natively perform, software instructions stored on a medium that makes the instructions readily executable by the hardware to specifically configure the hardware to perform the enumerated functions and specific computer operations described herein, a procedure or method for performing a function, or any combination of the above.

[0026] The description and claims may use the terms "a," "at least one of," and "one or more of" with reference to particular features and elements of exemplary embodiments. It should be understood that these terms and phrases are intended to state that there is at least one of a particular feature or element present in a particular exemplary embodiment, but that more than one may also be present. That is, these terms / phrases are not intended to limit the description or claims to the presence of a single feature / element or to require the presence of a plurality of such features / elements. To the contrary, these terms / phrases require only at least a single feature / element, and it is possible that a plurality of such features / elements may be present within the scope of the description and claims.

[0027] Furthermore, it should be understood that the use of the term “engine,” when used herein in connection with describing embodiments and features of the present invention, is not intended to limit a particular implementation for realizing and / or performing actions, steps, processes, etc., attributed to and / or performed by an engine. An engine may be, but is not limited to, software, hardware, and / or firmware, or any combination thereof, that performs the specified functions, including, but not limited to, the use of a general-purpose and / or special-purpose processor in combination with appropriate software loaded or stored in machine-readable memory and executed by the processor. Furthermore, names associated with particular engines, unless otherwise specified, are for convenience of reference and are not intended to limit a particular implementation. Furthermore, functionality attributed to an engine may be equivalently performed by multiple engines, incorporated into and / or combined with the functionality of other engines of the same or different type, or distributed across one or more engines in various configurations.

[0028] Furthermore, it should be understood that the following description uses multiple different examples of various elements of the exemplary embodiments to further explain exemplary implementations of the exemplary embodiments and to aid in understanding the mechanisms of the exemplary embodiments. These examples are intended to be non-limiting and do not exhaust the various possibilities for implementing the mechanisms of the exemplary embodiments. In light of this description, it will be apparent to those skilled in the art that there are many other alternative implementations of these various elements that could be utilized in addition to or in place of the examples provided herein without departing from the scope of the present invention.

[0029] As such, the exemplary embodiments may be utilized in many different types of data processing environments. To provide a context for the description of certain elements and functionality of the exemplary embodiments, Figures 1 and 2 are provided below as exemplary environments in which aspects of the exemplary embodiments may be implemented. It should be understood that Figures 1 and 2 are only examples, and are not intended to assert or imply any limitation with regard to the environments in which aspects or embodiments of the present invention may be implemented. Many modifications to the illustrated environments may be made without departing from the scope of the present invention.

[0030] 1 shows an illustration of an exemplary distributed data processing system in which aspects of the exemplary embodiments may be implemented. Distributed data processing system 100 may include a network of computers in which aspects of the exemplary embodiments may be implemented. Distributed data processing system 100 includes at least one network 102, which is the medium used to provide communications links between various interconnected devices and computers within distributed data processing system 100. Network 102 may include connections such as, for example, wires, wireless communication links, fiber optic cables, etc.

[0031] In the depicted example, servers 104 and 106 are connected to network 102 along with storage unit 108. Additionally, clients 110, 112, and 114 are also connected to network 102. Clients 110, 112, and 114 may be, for example, personal computers, network computers, etc. In the depicted example, server 104 provides data such as, for example, boot files, operating system images, and applications, i.e., execution environments, to clients 110, 112, and 114. Clients 110, 112, and 114 are clients of server 104 in the depicted example. Distributed data processing system 100 may include additional servers, clients, and other devices not shown.

[0032] In the depicted example, distributed data processing system 100 is the Internet, with network 102 representing a worldwide collection of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) suite of protocols to communicate with each other. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational, and other computer systems that route data and messages. Of course, distributed data processing system 100 may also be implemented to include a number of different types of networks, such as, for example, an intranet, a local area network (LAN), a wide area network (WAN), etc. As noted above, FIG. 1 is intended as an example, and not as an architectural limitation for different embodiments of the present invention, and thus the specific elements illustrated in FIG. 1 should not be considered limitations with regard to the environments in which illustrative embodiments of the present invention may be implemented.

[0033] As shown in FIG. 1 , one or more of the computing devices, e.g., server 104, may be specially configured to implement an intelligent execution environment identification mechanism. Configuring a computing device may include providing special-purpose hardware, firmware, etc. to facilitate performing the operations and generating output described herein with respect to the exemplary embodiments. Configuring a computing device may also, or alternatively, include providing a software application stored in one or more storage devices and loaded into memory of a computing device, such as server 104, to cause one or more hardware processors of the computing device to execute the software application, which configures the processor to perform the operations and generate output described herein with respect to the exemplary embodiments. Furthermore, any combination of special-purpose hardware, firmware, software applications running on hardware, etc. may be used without departing from the scope of the exemplary embodiments.

[0034] It should be understood that once a computing device is configured in one of these ways, the computing device becomes a dedicated computing device specifically configured to implement the mechanisms of the exemplary embodiments, and not a general-purpose computing device. Further, as described below, implementing the mechanisms of the exemplary embodiments improves the functionality of the computing device and provides useful, tangible results that facilitate intelligently identifying execution environments for the execution of computing jobs.

[0035] As described above, the mechanisms of the exemplary embodiments utilize specially configured computing devices or data processing systems to intelligently identify an execution environment for the execution of a computing job. These computing devices or data processing systems may include various hardware elements specially configured, either through hardware, software, or a combination of hardware and software, to implement one or more of the systems / subsystems described herein. FIG. 2 is a block diagram of just one example data processing system in which aspects of the exemplary embodiments may be implemented. Data processing system 200 is an example of a computer, such as server 104 of FIG. 1, in which computer-usable code or instructions implementing the processes and aspects of the exemplary embodiments of the present invention may be arranged and / or executed to achieve the operations, output, and external effects of the exemplary embodiments described herein.

[0036] In the depicted example, data processing system 200 utilizes a hub architecture including a north bridge and memory controller hub (NB / MCH) 202 and a south bridge and input / output (I / O) controller hub (SB / ICH) 204. A processing unit 206, main memory 208, and a graphics processor 210 are connected to NB / MCH 202. Graphics processor 210 may be connected to NB / MCH 202 via an accelerated graphics port (AGP).

[0037] In the illustrated example, a local area network (LAN) adapter 212 connects to the SB / ICH 204. An audio adapter 216, a keyboard and mouse adapter 220, a modem 222, a read-only memory (ROM) 224, a hard disk drive (HDD) 226, a CD-ROM drive 230, a universal serial bus (USB) port and other communication ports 232, and PCI / PCIe® devices 234 connect to the SB / ICH 204 via bus 238 and bus 240. PCI / PCIe® devices may include, for example, Ethernet adapters, add-in cards, and PC cards for notebook computers. PCI uses a card bus controller, while PCIe® does not. ROM 224 may be, for example, a flash basic input / output system (BIOS).

[0038] The HDD 226 and CD-ROM drive 230 connect to the SB / ICH 204 via a bus 240. The HDD 226 and CD-ROM drive 230 may use, for example, an Integrated Drive Electronics (IDE) or a Serial Advanced Technology Attachment (SATA) interface. A super I / O (SIO) device 236 may be connected to the SB / ICH 204.

[0039] An operating system runs on processing unit 206. The operating system coordinates and provides control over various components within data processing system 200 of FIG. 2. As a client, the operating system may be a commercially available operating system such as Microsoft® Windows® 10®. An object-oriented programming system, such as a Java®™ programming system, may operate in conjunction with the operating system to provide calls to the operating system from Java®™ programs or applications executing on data processing system 200.

[0040] As a server, data processing system 200 may be, for example, an IBM® eServer™ System p® computer system running the Advanced Interactive Executive (AIX®) operating system or the LINUX® operating system, a Power™ processor-based computer system, etc. Data processing system 200 may be a symmetric multiprocessor (SMP) system that includes multiple processors within processing unit 206. Alternatively, a single processor system may be utilized.

[0041] Instructions for the operating system, object-oriented programming system, and applications or programs may be located on a storage device, such as HDD 226, and loaded into main memory 208 for execution by processing unit 206. The processes of the exemplary embodiments of the present invention may be performed by processing unit 206 using computer-usable program code, which may be located, for example, in a memory, such as main memory 208, ROM 224, or in one or more peripheral devices 226 and 230.

[0042] A bus system, such as bus 238 or bus 240 shown in FIG. 2, may be comprised of one or more buses. Of course, a bus system may be implemented using any type of communications fabric or architecture that provides for a transfer of data between different components or devices attached to the fabric or architecture. A communications unit, such as modem 222 or network adapter 212 in FIG. 2, may include one or more devices used to transmit and receive data. A memory may be, for example, main memory 208, ROM 224, or a cache such as found in NB / MCH 202 in FIG. 2.

[0043] As noted above, in some exemplary embodiments, the mechanisms of the exemplary embodiments may be implemented as special purpose hardware, firmware, or the like, application software stored in a storage device such as HDD 226, loaded into a memory such as main memory 208, and executed by one or more hardware processors such as processing unit 206. Accordingly, the computing device illustrated in FIG. 2 is specially configured to implement the mechanisms of the exemplary embodiments and to perform the operations and generate output described herein with respect to the mechanisms for intelligently identifying an execution environment for the execution of a computing job.

[0044] Those skilled in the art will appreciate that the hardware in Figures 1 and 2 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical disk drives, may be used in addition to or in place of the hardware depicted in Figures 1 and 2. Additionally, the processes of the illustrative embodiments may be applied to multiprocessor data processing systems other than the SMP systems described above without departing from the scope of this invention.

[0045] Furthermore, data processing system 200 may take the form of any of several different data processing systems, including a client computing device, a server computing device, a tablet computer, a laptop computer, a telephone or other communication device, a personal digital assistant (PDA), etc. In some illustrative examples, data processing system 200 may be a portable computing device configured with flash memory to provide non-volatile memory for storing operating system files and / or user-generated data, for example. In essence, data processing system 200 may be any known or later-developed data processing system without architectural limitation.

[0046] The exemplary embodiments may be utilized in many different types of data processing environments. To provide a context for the description of certain elements and functionality of the exemplary embodiments, Figures 1 and 2 are provided as exemplary environments in which aspects of the exemplary embodiments may be implemented. It should be understood that Figures 1 and 2 are only examples and are not intended to assert or imply any limitation with regard to the environments in which aspects or embodiments of the invention may be implemented. Many modifications to the illustrated environments may be made without departing from the scope of the present invention.

[0047] According to one exemplary embodiment, Figures 1 and 2 are intended to illustrate an exemplary cognitive system for intelligently identifying an execution environment for the execution of computing jobs, which implements a request processing pipeline, a request processing methodology, and a request processing computer program product in which mechanisms of the exemplary embodiments are implemented. These requests may be provided as structured or unstructured request messages, natural language questions, or any other suitable format for requesting an action to be performed by a cognitive system. As described in more detail below, a particular application implemented in the cognitive system of the present invention is an application for intelligently identifying an execution environment for the execution of computing jobs.

[0048] In general terms, a cognitive system is a dedicated computer system or set of computer systems that consists of hardware and / or software logic (software combined with hardware logic on which software runs). Cognitive systems include artificial intelligence logic, such as natural language processing (NLP)-based logic, and machine learning logic, which may be provided as dedicated hardware, software running on hardware, or any combination of dedicated hardware and software running on hardware. The logic in a cognitive system implements cognitive operations, examples of which include, but are not limited to, request processing, identifying execution environments and their capabilities, and intelligent search algorithms.

[0049] 3 illustrates an example functional block diagram of a data processing system for intelligently identifying an execution environment for the execution of a computing job, according to one illustrative embodiment. Data processing system 300, such as data processing system 200 of FIG. 2 operating on a client device, such as client 110, 112, or 114 of FIG. 1, includes an intelligent execution environment identification mechanism 302 communicatively coupled to multiple execution environments 304a, 304b, ..., 304n, such as server 104 or 106 of FIG. 1. To identify an optimal execution environment from the multiple execution environments 304a-304n, intelligent execution environment identification mechanism 302 includes a machine learning model generation mechanism 306, an execution time estimation engine 314, an execution environment selection engine 316, and a machine learning model update engine 318.

[0050] When the intelligent execution environment identification mechanism 302 is initialized, the machine learning model generation mechanism 306 generates multiple machine learning models 320 for each of the multiple execution environments 304a-304n. Each machine learning model in the multiple machine learning models 320 for each execution environment in the multiple execution environments 304a-304n models the execution of a specific computing job in the execution environment based on characteristics of the input data for the specific computing job and characteristics of the execution environment. To generate each machine learning model for each execution environment, the execution context identification engine 308 in the machine learning model generation mechanism 306 collects execution context information for multiple test cases. The multiple test cases cover various input data that test all or a portion of the capabilities of the nodes in each execution environment to accurately test the functionality of the execution environment using all or a portion of the nodes that form the execution environment. To this end, the execution context identification engine 308 collects execution context information for each test case, which may include, but is not limited to, the size of the input data, the number of fields, the number of categorical fields, the number of continuous fields, the number of categories, univariate statistics for categorical fields such as the percentage of each category, univariate statistics for continuous fields such as minimum, maximum, mean, standard deviation, variance, and standard error, metadata, parameters, etc. Once collected, the execution context identification engine 308 may transform the execution context information, which may also be referred to as input data, as needed. In one embodiment, the execution context identification engine 308 may perform feature scaling and then count the number of values ​​within equal-width intervals. In other embodiments, the execution context identification engine 308 may perform binning to convert continuous variables into categorical variables. In one embodiment, the execution context identification engine 308 may perform merging to reduce the number of categories for categorical variables. The execution context identification engine 308 then identifies values ​​associated with each piece of execution context information to provide key parameters in the generation of the machine learning model.

[0051] Furthermore, to generate each machine learning model for each execution environment, the execution environment identification engine 310 in the machine learning model generation mechanism 306 collects execution environment information for each of the multiple execution environments 304a-304n. To this end, the execution environment identification engine 310 collects execution environment information for each of the test cases, which may include, but is not limited to, the number of processors, processor characteristics, memory size, software development information, operating system information, etc.

[0052] Using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments 304a-304n, the machine learning model generation engine 312 in the machine learning model generation mechanism 306 executes each test case in each of the plurality of execution environments 304a-304n that may execute that test case. That is, based on the capabilities of a particular execution environment and the requirements of the execution context information, one or more of the plurality of execution environments 304a-304n may not be able to execute a particular test case. As a result, a machine learning model for that particular execution environment is not established under that test case. However, for each execution environment in the plurality of execution environments 304a-304n that may execute a test case in the plurality of test cases, the machine learning model generation mechanism 306 executes the test case and records the execution time of the test case.

[0053] Once all test cases have been executed, the machine learning model generation engine 312 generates a set of machine learning models, such as linear regression models, regression tree models, etc., for the relationship between the execution time of the test cases in the execution environment and the execution context of the associated test cases. In one exemplary embodiment, the models are: Y=f(X) where Y is the execution time, X is the execution information (execution context information and execution environment information), and f(X) is the model. The machine learning model generation engine 312 generates one base machine learning model BM using all of the collected information, i.e., the execution context information, values ​​associated with each piece of execution context information, and all of the execution environment information, as predictors. In addition to generating one base machine learning model BM, the machine learning model generation engine 312 generates N light machine learning models LM using a portion of the collected information, i.e., some but not all of the execution context information, values ​​associated with each piece of execution context information, and the execution environment information, as predictors (e.g., when N = 5, the machine learning model generation engine 312 generates LM1, LM2, LM3, LM4, and LM5). In one exemplary embodiment, the number N may be determined by taking the square root of the number of predictors. In one exemplary embodiment, the portion of the collected information may include fixed primary predictors such as the number of processors and memory size, and optional predictors such as metadata and parameters.

[0054] Base machine learning model BM and light machine learning models LM1 to LM N For each of the weights, the machine learning model generation engine 312 assigns an initial weight WBM to the BM and an initial weight LM N WLM N In one example, the machine learning model generation engine 312 may assign weights as follows: ·WBM=w1 where 0 <w1<1 WLMn=(1-w1) / N In this example, with N=5 and w1=0.5, the weights are: WBM=0.5 WLM1=0.1 WLM2=0.1 WLM3=0.1 WLM4=0.1 WLM5=0.1 In other exemplary embodiments, the machine learning model generation engine 312 may assign or modify weights based on model accuracy. The machine learning model generation engine 312 stores each of the machine learning models as a machine learning model 320 in storage 322.

[0055] Furthermore, the intelligent execution environment identification mechanism 302 first intelligently identifies an execution environment for the execution of a computing job, based on each of the machine learning models 320. Therefore, when a user wants to run a new computing job, the execution time estimation engine 314 collects execution context information about the new computing job, which may include, but is not limited to, the size of the input data, the number of fields, the number of categorical fields, the number of continuous fields, the number of categories, univariate statistics of categorical fields (e.g., the percentage of each category), univariate statistics of continuous fields (e.g., minimum, maximum, mean, standard deviation, variance, standard error), metadata, parameters, etc. The execution time estimation engine 314 then identifies values ​​associated with each piece of execution context information about the new computing job to provide key parameters for generating the machine learning model. The execution time estimation engine 314 also collects execution environment information about each of the multiple execution environments 304a-304n, which may include, but is not limited to, the number of processors, processor characteristics, memory size, software development information, operating system information, etc.

[0056] For each execution environment among the plurality of execution environments 304a-304n, the execution time estimation engine 314 predicts the execution environment's execution time by applying particular ones of the machine learning models 320 that match the execution context information and key parameters of the new computing job and the execution environment information of the associated execution environment to determine a predicted execution time ETB for the particular one of the base machine learning models of the associated execution environment and a predicted execution time ETLn for the particular one of the light machine learning models of the associated execution environment. For each execution environment, the execution time estimation engine 314 aggregates the predicted execution times of the base machine learning model and the light machine learning model to provide an aggregated execution time for the execution environment as follows:

number

[0057] Once the execution time estimation engine 314 determines the aggregate execution time for each of the execution environments 304a-304n, the execution environment selection engine 316 summarizes the execution time for the new computing job in each execution environment. In one exemplary embodiment, the execution environment selection engine 316 presents the summary of execution times to a user in a user interface so that the user may select an appropriate execution environment for the new computing job. In another exemplary embodiment, the execution environment selection engine 316 selects an appropriate execution environment for executing the new computing job based on predefined user settings, i.e., a set of rules identified by the user prior to requesting intelligent identification of an execution environment for execution of the computing job. Once the user provides a selection of an execution environment or the execution environment selection engine 316 selects an execution environment based on the predefined user settings, the execution environment selection engine 316 executes the new computing job in the selected execution environment and collects relevant data during execution.

[0058] The associated data collected about the selected execution environment may indicate changes in the execution environment since the test case was run or since the last user-submitted computing job was run, such as differences in hardware (e.g., virtual machine vs. physical hardware, hard disk drive vs. solid-state drive), optimizations, dependency library updates (e.g., central processing unit vs. graphics processing unit), hardware replacements and improvements, etc. Thus, possible changes in the execution environment cause the machine learning model update engine 318 to retrain the machine learning models associated with the selected execution environment. The machine learning model update engine 318 updates both the base machine learning model for the associated execution environment and each of the lite machine learning models for the associated execution environment.

[0059] To update the base machine learning model of the associated execution environment and each of the lite machine learning models of the associated execution environment, the machine learning model update engine 318 adds relevant data collected about the selected execution environment, i.e., collected execution context information about the new computing job, key parameters, execution environment information about the selected execution environment, and observed execution time, to a data cache. The machine learning model update engine 318 determines whether the number of records in the data cache is greater than a first predefined threshold S1. If the number of records in the data cache is greater than S1, the machine learning model update engine 318 calculates the model accuracy R based on the cached data. 1 The machine learning model update engine 318 then calculates the calculated accuracy (R 1 ) and retrain each Wright machine learning model by adjusting the weights of the Wright model based on the

number

[0060] To ensure that the Wright models are always fresh based on their accuracy, the machine learning model update engine 318 continues to calculate the accuracy of each "in use" Wright model based on the cached data. The machine learning model update engine 318 also builds new Wright machine learning models based on the cached data and updates them based on a first predefined threshold (T acc1 ) caches light machine learning models that have accuracy higher than a first predefined threshold (T acc1 ), the machine learning model update engine 318 retrieves all "in use" and cached light machine learning models, sorts the blended light machine learning models by accuracy, and selects the top N light machine learning models, which are then used as the new set of "in use" light machine learning models. The machine learning model update engine 318 may retrain the new set of "in use" light machine learning models by reusing the previously determined weights for the new set of "in use" light machine learning models or by assigning new weights to the selected light machine learning models using the weight adjustment formula described above.

[0061] With respect to the base machine learning model, the number of records in the cached data is greater than a second predefined threshold S2, and the accuracy of the base model is greater than the second predefined threshold (T acc2), the machine learning model update engine 318 generates a new base machine learning model using the cached data. In this case, the machine learning model update engine 318 may retrain each of the base machine learning model and the lite machine learning model associated with the particular execution environment by assigning new weights to the selected machine learning model using the weight adjustment formula described above, or using a formula previously used for determining the initial weights used during the initial training of the base machine learning model and the lite machine learning model for the particular execution environment.

[0062] In this way, the intelligent execution environment identification mechanism 302 improves accuracy in intelligently identifying the execution environment for each subsequent computing job execution by retraining the machine learning model after each computing job execution, thereby providing an improvement over traditional execution environment selection. The intelligent execution environment identification mechanism 302 is also easy to use because it does not require domain knowledge or experience. Furthermore, the intelligent execution environment identification mechanism 302 is scalable because each execution environment node has its own distinct set of machine learning models, i.e., a base machine learning model and a lite machine learning model.

[0063] The present invention may be a system, a method, or a computer program product, or a combination thereof. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0064] A computer-readable storage medium may be any tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick®, floppy® disk, mechanically encoded devices such as punch cards or grooved ridge structures having instructions recorded thereon, and any suitable combination thereof. Computer-readable storage medium, as used herein, should not be construed as a transitory signal per se, such as, for example, radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted over wires.

[0065] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, fiber optic transmission cables, wireless transmission cables, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0066] The computer-readable program instructions for carrying out the operations of the present invention may be source or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java®, Smalltalk®, C++, and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the last scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or a connection may be made to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing state information of the computer readable program instructions to execute the computer readable program instructions to perform aspects of the present invention.

[0067] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0068] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to create a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium, directing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored constitutes an article of manufacture including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0069] Furthermore, the computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device to create a computer-implemented process and cause the computer, other programmable apparatus, or other device to perform a series of operational steps, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0070] FIG. 4 illustrates an example flow diagram of operations performed by an intelligent execution environment identification mechanism, such as intelligent execution environment identification mechanism 302 of FIG. 3, when identifying an execution environment for the execution of a new computing job, according to one exemplary embodiment. When operations begin, when a user submits a request to execute a new computing job, the execution time estimation engine of the intelligent execution environment identification mechanism collects execution context information about the new computing job (step 402). The execution context information may include, but is not limited to, the size of the input data, the number of fields, the number of categorical fields, the number of continuous fields, the number of categories, univariate statistics of categorical fields such as the percentage of each category, univariate statistics of continuous fields such as the minimum value, maximum value, mean value, standard deviation, variance, and standard error, metadata, parameters, etc. The execution time estimation engine then identifies values ​​associated with each piece of execution context information about the new computing job to provide key parameters in generating a machine learning model (step 404). The execution time estimation engine also collects execution environment information about each of a plurality of execution environments (step 406). Execution environment information may include, but is not limited to, the number of processors, processor characteristics, memory size, software development information, operating system information, etc.

[0071] For each execution environment of the plurality of execution environments, the execution time estimation engine predicts the execution environment's execution time (step 408). The execution time estimation engine does so by applying particular ones of a set of existing machine learning models that match the execution context information and key parameters of the new computing job and the execution environment information of the associated execution environment to determine a predicted execution time ETB for particular ones of the base machine learning models of the associated execution environment and a predicted execution time ETLn for particular ones of the light machine learning models of the associated execution environment. For each execution environment, the execution time estimation engine aggregates the predicted execution times of the base machine learning model and the light machine learning model to provide an aggregated execution time for the execution environment (step 410), as follows:

number

[0072] Once the execution time estimation engine determines the aggregate execution times for each of the multiple execution environments, the execution environment selection engine summarizes the execution times for the new computing job in each execution environment (step 412). In one exemplary embodiment, the execution environment selection engine presents the summary of execution times to the user in a user interface so that the user may select an appropriate execution environment for the new computing job. In another exemplary embodiment, the execution environment selection engine selects an appropriate execution environment for executing the new computing job based on predefined user settings, i.e., a set of rules identified by the user prior to requesting intelligent identification of an execution environment for execution of the computing job. Once the user provides a selection of the execution environment or the execution environment selection engine selects an execution environment based on the predefined user settings, the execution environment selection engine executes the new computing job in the selected execution environment (step 414) and collects relevant data during execution (step 416), after which the operation terminates.

[0073] 5 illustrates an example flow diagram of operations performed in generating a set of existing machine learning models to be used by the execution time estimation engine of the intelligent execution environment identification mechanism in predicting the execution time of an execution environment, according to one exemplary embodiment. Operation begins when an execution context identification engine within the machine learning model generation mechanism of the intelligent execution environment identification mechanism collects execution context information for a plurality of test cases (step 502). The collected execution context information for each of the test cases may include, but is not limited to, the size of the input data, the number of fields, the number of categorical fields, the number of continuous fields, the number of categories, univariate statistics for categorical fields such as the percentage of each category, univariate statistics for continuous fields such as the minimum value, maximum value, mean value, standard deviation, variance, standard error, metadata, parameters, etc. The execution context identification engine then identifies values ​​associated with each piece of execution context information to provide key parameters in generating the machine learning model (step 504).

[0074] Furthermore, an execution environment identification engine within the machine learning model generation mechanism of the intelligent execution environment identification mechanism collects execution environment information for each of the multiple execution environments (step 506). The collected execution environment information for each of the test cases may include, but is not limited to, the number of processors, processor characteristics, memory size, software development information, operating system information, etc.

[0075] Using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments, a machine learning model generation engine in the machine learning model generation mechanism of the intelligent execution environment identification mechanism executes each test case in the plurality of test cases in each of the plurality of execution environments (step 508). Once all the test cases have been executed, the machine learning model generation engine generates a set of machine learning models, such as linear regression models, regression tree models, etc., for the relationship between the execution times of the test cases in the execution environments and the execution contexts of the associated test cases (step 510). In one exemplary embodiment, the models are: Y=f(X) where Y is the execution time, X is the execution information (execution context information and execution environment information), and f(X) is the model. The machine learning model generation engine uses all of the collected information, i.e., the execution context information, values ​​associated with each piece of execution context information, and all of the execution environment information, as predictors to generate one base machine learning model BM. In addition to generating one base machine learning model BM, the machine learning model generation engine uses a portion of the collected information, i.e., some but not all of the execution context information, values ​​associated with each piece of execution context information, and the execution environment information, as predictors to generate N light machine learning models LM (e.g., when N = 5, the machine learning model generation engine 312 generates LM1, LM2, LM3, LM4, and LM5). In one exemplary embodiment, the number N may be determined by taking the square root of the number of predictors. In one exemplary embodiment, the portion of the collected information may include fixed primary predictors such as the number of processors and memory size, and optional predictors such as metadata and parameters.

[0076] Base machine learning model BM and light machine learning models LM1 to LM N For each of the BMs, the machine learning model generation engine assigns an initial weight WBM to the BMs and an initial weight LM NWLM N (step 512). In one example, the machine learning model generation engine 312 may assign weights as follows: ·WBM=w1 where 0 <w1<1 WLMn=(1-w1) / N In this example, with N=5 and w1=0.5, the weights are: WBM=0.5 WLM1=0.1 WLM2=0.1 WLM3=0.1 WLM4=0.1 WLM5=0.1 In other exemplary embodiments, the machine learning model generation engine may assign or modify weights based on model accuracy. The machine learning model generation engine stores each of the machine learning models in storage as a machine learning model (step 514), with the operations terminating thereafter.

[0077] FIG. 6 illustrates an example flow diagram of operations performed by a machine learning model update engine when updating a machine learning model utilized by an execution environment selection engine when identifying an execution environment for the execution of a new computing job, according to one exemplary embodiment. Again, the relevant data collected about the selected execution environment in step 416 of FIG. 4 may indicate changes in the execution environment since the test case was run or since the last user-submitted computing job was run, such as hardware differences (e.g., virtual machine vs. physical hardware, hard disk drive vs. solid-state drive), optimizations, dependency library updates (e.g., central processing unit vs. graphics processing unit), hardware replacements and improvements, etc. Therefore, to account for possible changes in the execution environment, upon initiation of operation, the machine learning model update engine adds the relevant data collected about the selected execution environment, i.e., the collected execution context information for the new computing job, key parameters, execution environment information about the selected execution environment, and observed execution time, to a data cache (step 602). The machine learning model update engine then determines whether the number of records in the data cache is greater than a first predetermined value (step 604). In step 604, if the number of records in the data cache is not greater than the first predetermined value, the operation ends. In step 604, if the number of records in the data cache is greater than the first predetermined value, the machine learning model update engine updates the model accuracy R based on the cached data. 1 (step 606). The machine learning model update engine then calculates the calculated accuracy (R 1 ) and retrain each light machine learning model by adjusting the light model weights based on the light model weights (step 608).

number

[0078] With respect to the base machine learning model, the machine learning model update engine determines whether the number of records in the data cache is greater than a second predetermined value (step 610). If the number of records in the data cache is not greater than the second predetermined value in step 610, the operation ends. If the number of records in the data cache is greater than the second predetermined value in step 610, the machine learning model update engine determines whether the accuracy of the base model is greater than a second predefined threshold (T acc2 ) (step 612). In step 612, it is determined whether the accuracy of the base model is less than a second predefined threshold (T acc2 ), the operation ends. In step 612, the accuracy of the base model is determined to be greater than or equal to a second predefined threshold (T acc2 ), the machine learning model update engine generates a new base machine learning model using the cached data (step 614). The machine learning model update engine then replaces the existing base machine learning model for the particular execution environment with the newly generated base machine learning model for the particular execution environment in the set of machine learning models (step 616). The machine learning model update engine may also retrain each of the newly generated base machine learning models and lite machine learning models associated with the particular execution environment by assigning new weights to the selected machine learning models using the weight adjustment formula described above, or using a formula previously used for determining the initial weights used during the initial training of the base machine learning model and lite machine learning model for the particular execution environment (step 618), after which the operation terminates.

[0079] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order depicted. For example, depending on the functionality involved, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may possibly be executed in the reverse order. It will also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

[0080] Thus, an exemplary embodiment provides a mechanism for intelligently identifying an execution environment for executing a computing job. The mechanism constructs a machine learning model to reflect the relationship between the job's execution time and execution context, e.g., the execution environment, input data information, key parameters, etc. Then, prior to the actual execution of the computing job, the mechanism estimates the execution time for executing the execution flow of the computing job in each candidate execution environment among the plurality of execution environments based on the machine learning model of each execution environment among the plurality of execution environments, to provide several combinations of execution environment and execution time as options for selection. That is, the mechanism may notify a user of the job's scale (heavy or light) and the estimated execution time in all available candidate execution environments, allowing the user to select an appropriate execution environment based on performance, or the mechanism may automatically select an execution environment based on predefined user settings. After the execution of each computing job in the selected execution environment, the mechanism collects execution information and refreshes the associated machine learning model.

[0081] As noted above, it should be understood that the exemplary embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment containing both hardware and software elements. In one exemplary embodiment, the mechanisms of the exemplary embodiments are implemented in software or program code, including but not limited to firmware, resident software, microcode, etc.

[0082] A data processing system suitable for storing and / or executing program code includes at least one processor coupled directly or indirectly to memory elements via a communication bus, such as a system bus. The memory elements may include local memory utilized during the actual execution of the program code, bulk storage, and cache memory that provides temporary storage of at least some program code to reduce the number of times the code needs to be retrieved from bulk storage during execution. The memory may be of various types, including but not limited to ROM, PROM, EPROM, EEPROM, DRAM, SRAM, flash memory, solid-state memory, etc.

[0083] Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system directly or through intervening wired or wireless I / O interfaces and / or controllers, etc. I / O devices can take many different forms other than traditional keyboards, displays, pointing devices, etc., such as communication devices coupled via wired or wireless connections, including but not limited to smart phones, tablet computers, touch screen devices, voice recognition devices, etc. Any known or later-developed I / O devices are intended to be within the scope of the exemplary embodiments.

[0084] Network adapters may also be coupled to the system to enable the data processing system to be coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet® cards are just a few of the currently available types of network adapters for wired communications. Wireless communication-based network adapters may also be utilized, including, but not limited to, 802.11a / b / g / n wireless communication adapters, Bluetooth® wireless adapters, and the like. Any known or later-developed network adapter is intended to be within the scope of the present invention.

[0085] The description of the present invention has been presented for purposes of illustration and description and is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The embodiments have been chosen and described to best explain the principles and practical applications of the invention and to enable those skilled in the art to understand the invention in various embodiments with various modifications suited to the particular uses contemplated. The terms used herein have been chosen to best explain the principles, practical applications, or technical improvements over commercially available technologies of the embodiments, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A method for intelligently identifying an execution environment for executing a computing job in a data processing system, comprising: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Including, wherein the set of existing machine learning models is: executing each test case in the plurality of test cases in each execution environment of the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generating the set of machine learning models based on a relationship between an execution time of each test case in an associated execution environment and an execution context of the associated test case; Created by method.

2. 1. A method for intelligently identifying an execution environment for executing a computing job in a data processing system, comprising: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Including, The set of existing machine learning models includes: Collecting execution context information for a plurality of test cases; identifying key parameters using values ​​associated with each of the execution context information for the plurality of test cases; collecting execution environment information regarding each of the plurality of execution environments; executing each test case in the plurality of test cases in each execution environment of the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generating the set of machine learning models based on a relationship between an execution time of each test case in an associated execution environment and an execution context of the associated test case; assigning an initial weight to each machine learning model in the set of machine learning models; produced by the method comprising method.

3. 1. A method for intelligently identifying an execution environment for executing a computing job in a data processing system, comprising: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Including, Updating the set of machine learning models utilized in identifying an execution environment for executing the computing job includes: adding the associated data collected for the selected execution environment to a data cache; in response to the number of records in the data cache being greater than a first predetermined value; Calculating model accuracy based on the cached data; retraining the at least one light machine learning model in the set of machine learning models by adjusting a weight of the light machine learning model based on the calculated model accuracy; in response to the number of records in the data cache being greater than a second predetermined value; determining whether the accuracy of a base model in the set of machine learning models is less than a predefined threshold (T acc ); generating a new base machine learning model using the cached data in response to the accuracy of the base model being less than the predefined threshold (T acc ); replacing an existing base machine learning model for an associated execution environment with a newly generated base machine learning model for the associated execution environment in the set of machine learning models; Including, method.

4. 1. A method for intelligently identifying an execution environment for executing a computing job in a data processing system, comprising: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Including, The set of machine learning models for each execution environment among the plurality of execution environments includes: a base machine learning model that uses all of the execution context information of a plurality of test cases, values ​​associated with each of the execution context information of the test cases, and all related execution environment information; at least one light machine learning model that uses some but not all of the execution context information of the test cases, values ​​associated with each of the execution context information of the test cases, and all relevant execution environment information; Including, method.

5. 5. The method of claim 1, wherein the selection of the execution environment from the plurality of execution environments based on the summary of aggregated predicted execution times of the computing jobs is either a user selection or an automatic selection based on predefined user settings.

6. 5. The method of claim 1, wherein the execution context information is selected from the group comprising: size of input data, number of fields, number of categorical fields, number of continuous fields, univariate statistics of categorical fields, univariate statistics of continuous fields, metadata, and parameters.

7. The method of any one of claims 1 to 4, wherein the key parameters are identified based on values ​​associated with the execution context information for the computing job.

8. The method of any one of claims 1 to 4, wherein the execution environment information is selected from the group consisting of number of processors, processor characteristics, memory size, software development information, and operating system information.

9. retraining each of the newly generated base machine learning model and the lite machine learning model associated with the execution environment by assigning new weights to them; The method of claim 3 further comprising:

10. 1. A computer program for intelligently identifying an execution environment for executing a computing job, the computer program comprising: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Execute wherein the set of existing machine learning models is: executing each test case in the plurality of test cases in each execution environment of the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generating the set of machine learning models based on a relationship between an execution time of each test case in an associated execution environment and an execution context of the associated test case; Created by Computer program.

11. 1. A computer program for intelligently identifying an execution environment for executing a computing job, the computer program comprising: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Execute The set of existing machine learning models is then processed by the computer. Collecting execution context information for a plurality of test cases; identifying key parameters using values ​​associated with each of the execution context information for the plurality of test cases; collecting execution environment information regarding each of the plurality of execution environments; executing each test case in the plurality of test cases in each execution environment of the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generating the set of machine learning models based on a relationship between an execution time of each test case in an associated execution environment and an execution context of the associated test case; assigning an initial weight to each machine learning model in the set of machine learning models; is generated by further executing Computer program.

12. 1. A computer program for intelligently identifying an execution environment for executing a computing job, the computer program comprising: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Execute The computer program causes the computer to: update the set of machine learning models used in identifying an execution environment for executing the computing job; adding the associated data collected for the selected execution environment to a data cache; in response to the number of records in the data cache being greater than a first predetermined value; Calculating model accuracy based on the cached data; retraining the at least one light machine learning model in the set of machine learning models by adjusting a weight of the light machine learning model based on the calculated model accuracy; in response to the number of records in the data cache being greater than a second predetermined value; determining whether the accuracy of a base model in the set of machine learning models is less than a predefined threshold (T acc ); generating a new base machine learning model using the cached data in response to the accuracy of the base model being less than the predefined threshold (T acc ); replacing an existing base machine learning model for an associated execution environment with a newly generated base machine learning model for the associated execution environment in the set of machine learning models; by further executing Computer program.

13. 1. A computer program for intelligently identifying an execution environment for executing a computing job, the computer program comprising: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Execute The set of machine learning models for each execution environment among the plurality of execution environments includes: a base machine learning model that uses all of the execution context information of a plurality of test cases, values ​​associated with each of the execution context information of the test cases, and all related execution environment information; at least one light machine learning model that uses some but not all of the execution context information of the test cases, values ​​associated with each of the execution context information of the test cases, and all relevant execution environment information; Including, Computer program.

14. 14. The computer program product of claim 10, wherein the selection of the execution environment from the plurality of execution environments based on the summary of aggregated predicted execution times of the computing jobs is either a user selection or an automatic selection based on predefined user settings.

15. the execution context information is selected from the group consisting of input data size, number of fields, number of categorical fields, number of continuous fields, univariate statistics of categorical fields, univariate statistics of continuous fields, metadata, and parameters; the key parameters are identified based on values ​​associated with the execution context information for the computing job; the execution environment information is selected from the group consisting of number of processors, processor characteristics, memory size, software development information, and operating system information; A computer program according to any one of claims 10 to 13.

16. The computer program causes the computer to: retraining each of the newly generated base machine learning model and the lite machine learning model associated with the execution environment by assigning new weights to them; The computer program product of claim 12 , further comprising:

17. 1. An apparatus for intelligently identifying an execution environment for executing a computing job, comprising: a processor; a memory coupled to the processor; the memory containing instructions that, when executed by the processor, cause the processor to: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Execute wherein the set of existing machine learning models is: executing each test case in the plurality of test cases in each execution environment of the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generating the set of machine learning models based on a relationship between an execution time of each test case in an associated execution environment and an execution context of the associated test case; Created by Device.

18. 1. An apparatus for intelligently identifying an execution environment for executing a computing job, comprising: a processor; a memory coupled to the processor; the memory containing instructions that, when executed by the processor, cause the processor to: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Execute The set of existing machine learning models is generated by the instructions to the processor: Collecting execution context information for a plurality of test cases; identifying key parameters using values ​​associated with each of the execution context information for the plurality of test cases; collecting execution environment information regarding each of the plurality of execution environments; executing each test case in the plurality of test cases in each execution environment of the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generating the set of machine learning models based on a relationship between an execution time of each test case in an associated execution environment and an execution context of the associated test case; assigning an initial weight to each machine learning model in the set of machine learning models; is generated by further executing Device.

19. 1. An apparatus for intelligently identifying an execution environment for executing a computing job, comprising: a processor; a memory coupled to the processor; the memory containing instructions that, when executed by the processor, cause the processor to: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Execute The instructions cause the processor to update the set of machine learning models utilized in identifying an execution environment for executing the computing job. adding the associated data collected for the selected execution environment to a data cache; in response to the number of records in the data cache being greater than a first predetermined value; Calculating model accuracy based on the cached data; retraining the at least one light machine learning model in the set of machine learning models by adjusting a weight of the light machine learning model based on the calculated model accuracy; in response to the number of records in the data cache being greater than a second predetermined value; determining whether the accuracy of a base model in the set of machine learning models is less than a predefined threshold (T acc ); generating a new base machine learning model using the cached data in response to the accuracy of the base model being less than the predefined threshold (T acc ); replacing an existing base machine learning model for an associated execution environment with a newly generated base machine learning model for the associated execution environment in the set of machine learning models; by further executing Device.

20. 1. An apparatus for intelligently identifying an execution environment for executing a computing job, comprising: a processor; a memory coupled to the processor; the memory containing instructions that, when executed by the processor, cause the processor to: For each execution environment among the plurality of execution environments, predicting an execution time of the computing job in the execution environment by applying a set of existing machine learning models that match execution context information and key parameters of the computing job and execution environment information of the execution environment; aggregating predicted execution times of the machine learning models to form an aggregated predicted execution time for the computing jobs in the execution environment; aggregating the aggregated predicted execution times of the computing jobs for the plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of the aggregated predicted execution times of the computing job, executing the computing job in a selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Execute The set of machine learning models for each execution environment among the plurality of execution environments includes: a base machine learning model that uses all of the execution context information of a plurality of test cases, values ​​associated with each of the execution context information of the test cases, and all related execution environment information; at least one light machine learning model that uses some but not all of the execution context information of the test cases, values ​​associated with each of the execution context information of the test cases, and all relevant execution environment information; Including, Device.

21. 21. The apparatus of claim 17, wherein the selection of the execution environment from the plurality of execution environments based on the summary of aggregated predicted execution times of the computing jobs is either a user selection or an automatic selection based on predefined user settings.

22. the execution context information is selected from the group consisting of input data size, number of fields, number of categorical fields, number of continuous fields, univariate statistics of categorical fields, univariate statistics of continuous fields, metadata, and parameters; the key parameters are identified based on values ​​associated with the execution context information for the computing job; the execution environment information is selected from the group consisting of number of processors, processor characteristics, memory size, software development information, and operating system information; An apparatus according to any one of claims 17 to 20.

23. The instructions cause the processor to: retraining each of the newly generated base machine learning model and the lite machine learning model associated with the execution environment by assigning new weights to them; 20. The apparatus of claim 19, further comprising:

24. 1. A method for intelligently identifying an execution environment for executing a computing job in a data processing system, comprising: executing a set of machine learning models based on execution context information to predict an execution time of a computing job, and aggregating predicted execution times from the set of existing machine learning models to determine an aggregated predicted execution time of the computing job in each execution environment of a plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of aggregated predicted execution times of the computing job, executing the computing job in the selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Including, wherein the set of existing machine learning models is: executing each test case in the plurality of test cases in each execution environment of the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generating the set of machine learning models based on a relationship between an execution time of each test case in an associated execution environment and an execution context of the associated test case; Created by method.

25. 1. A computer program for intelligently identifying an execution environment for executing a computing job, the computer program comprising: executing a set of machine learning models based on execution context information to predict an execution time of a computing job, and aggregating predicted execution times from the set of machine learning models to determine an aggregated predicted execution time of the computing job in each execution environment of a plurality of execution environments; responsive to selecting an execution environment from the plurality of execution environments based on a compilation of aggregated predicted execution times of the computing job, executing the computing job in the selected execution environment; collecting relevant data during execution of the computing job in the selected execution environment; Execute wherein the set of machine learning models is: executing each test case in the plurality of test cases in each execution environment of the plurality of execution environments using the collected execution context information for each of the plurality of test cases and the execution environment information for each of the plurality of execution environments; generating the set of machine learning models based on a relationship between an execution time of each test case in an associated execution environment and an execution context of the associated test case; Created by Computer program.

Images