Elastic execution of machine learning workloads with application-based profiling

Abstract

Embodiments relate to systems, programs, and methods for supporting elastic execution of machine learning (ML) workloads using application-based profiling. A combined profile consisting of machine learning (ML) application execution and resource usage is generated. One or more features and one or more signatures are identified from the generated combined profile, and an ML execution model for the ML application execution and resource usage is constructed. The ML execution model is applied to utilize the identified one or more features and one or more signatures to provide one or more instructions to subsequent application execution. Application of the ML execution model supports and enables ML execution to elastically allocate and request one or more resources from a resource management component, with the elastic allocation supporting the application execution.

Description

[Technical field]

[0001] The present embodiments relate to training a machine learning model to profile the execution of a machine learning workload. More particularly, the present embodiments relate to generating a joint profile of application execution and resource usage, and utilizing the execution model encapsulated in the trained machine learning model to interact with a resource management system to support machine learning application execution and corresponding performance. [Background technology]

[0002] Artificial intelligence (AI) refers to the field of computer science directed at computers and computer behavior in relation to humans. AI refers to intelligence that allows a machine to make decisions based on information that maximize the chances of success in a given topic. More specifically, AI can learn from datasets to solve problems and provide relevant recommendations. For example, in the field of artificial intelligence computer systems, natural language systems (such as IBM® Watson® artificial intelligence computer systems or other natural language question answering systems) process natural language based on knowledge acquired by the system. To process natural language, the system may be trained with data derived from a corpus of knowledge, but the results produced may have different degrees of accuracy for various reasons.

[0003] Machine learning (ML), a subset of artificial intelligence (AI), utilizes algorithms to learn from data and generate predictions based on this data. ML is the application of AI through the creation of models, including neural networks, that can demonstrate learning behaviors by performing tasks that are not explicitly programmed. ML workloads require large data sets, fast and parallel access to data, and training algorithms to support learning. Examples of hardware supporting ML workloads include, but are not limited to, accelerators such as graphic processing units (GPUs), central processing unit (CPU) infrastructure, and storage solutions. Characterization of resource usage for ML workloads typically involves resource profiling, which is collected for debugging and execution timing studies. Resource profiling collection currently uses a timing framework that is independent of the application or workload execution. To improve correlation between workload execution and resource usage, it would be advantageous to have joint profiling of workload execution and resource usage using the same timing framework. Additionally, joint profiling would allow fine-grained correlation, such as at the task level or iteration level, with different sets of ML parameters. Thus, with an appropriate amount of resources to support an ML workload, joint profiling can be a means to support effective and efficient resource usage and workload execution. Summary of the Invention

[0004] Embodiments include systems, computer program products, and methods for employing an artificial intelligence platform to blend effective and efficient ML application execution with computational resource allocation.

[0005] In one embodiment, a system is provided for use with an artificial intelligence (AI) platform to train and maintain an ML application execution modeler and elastically execute machine learning workloads using application-based profiling. As shown, a system is provided having a processing unit, e.g., a processor, operably coupled to a memory and an AI platform in communication with the processing unit. The AI ​​platform is embedded with tools in the form of a profile manager, an ML manager, and a director. The profile manager is operative to generate a combined profile comprised of both ML application execution data and resource usage data. The ML manager is operative to identify one or more features and signatures from the combined profile and build an ML execution model for ML application execution and resource usage. The ML execution model utilizes the feature(s) and signature(s). The director is operative to apply the ML execution model and provide one or more instructions to subsequent application execution. The application by the director supports and enables the ML execution to elastically allocate and request one or more resources from the resource management component, with the elastic allocation supporting the application execution.

[0006] In another embodiment, a computer program product is provided that supports elastic execution of ML workloads using application-based profiling. The computer program product includes a computer-readable storage medium having program code embodied therein, the program code being executable by a processor to train and maintain an ML application execution modeler to elastically execute ML workloads using application-based profiling. The program code is provided for generating a combined profile comprised of both ML application execution data and resource usage data. The program code is provided for identifying one or more features and signatures from the combined profile and constructing an ML execution model for ML application execution and resource usage. The ML execution model utilizes the feature(s) and signature(s). The program code applies the ML execution model and provides one or more instructions to subsequent application execution. Application of the program code supports and enables the ML execution to elastically allocate and request one or more resources from a resource management component while the elastic allocation supports the execution of the application.

[0007] In yet another embodiment, a method is provided for supporting elastic execution of ML workloads using application-based profiling, including training and maintaining an ML execution model for elastically executing ML workloads using application-based profiling. A combined profile is generated that is comprised of both ML application execution data and resource usage data. One or more features and one or more signatures from the combined profile are identified, and an ML execution model for ML application execution and resource usage is constructed. The ML execution model utilizes the feature(s) and signature(s). The ML execution model is applied to provide one or more instructions to subsequent application execution. Application of the ML execution model supports and enables ML execution to elastically allocate and request one or more resources from a resource management component while the elastic allocation supports application execution.

[0008] According to one aspect, a computer system is provided that includes a processing unit operatively coupled to a memory and an artificial intelligence (AI) platform in communication with the processing unit having tools for elastically executing one or more machine learning workloads using application based profiling, where the tools include a profile manager that generates a combined profile consisting of machine learning (ML) application execution and resource usage, a machine learning manager that identifies one or more features and one or more signatures from the generated combined profile and builds an ML execution model for ML application execution performance and resource usage, where the ML execution model utilizes the identified one or more features and one or more signatures, and a director that applies the ML execution model and provides one or more directives to subsequent application execution including elastically allocating and requesting one or more resources from a resource management component to support the application execution.

[0009] According to another aspect, there is provided a computer program product for supporting elastic execution of machine learning workloads using application based profiling, the computer program product including a computer readable storage medium having program code embodied therein, the program code being executable by a processor to: generate a combined profile consisting of machine learning (ML) application execution and resource usage; identify one or more features and one or more signatures from the generated combined profile and construct an ML execution model for ML application execution performance and resource usage, the ML execution model utilizing the identified one or more features and one or more signatures; and apply the ML execution model and provide one or more instructions to subsequent application execution including elastically allocating and requesting one or more resources from a resource management component to support the application execution.

[0010] According to another aspect, a method is provided, the method including generating a combined profile comprised of machine learning (ML) application execution and resource usage, identifying one or more features and one or more signatures from the generated combined profile and constructing an ML execution model for ML application execution performance and resource usage, the ML execution model utilizing the identified one or more features and one or more signatures, and applying the ML execution model to provide one or more directives to subsequent application execution including elastically allocating and requesting one or more resources from a resource management component to support the application execution.

[0011] These and other features and advantages will become apparent from the following detailed description of the presently preferred embodiments taken in conjunction with the accompanying drawings.

[0012] Preferred embodiments of the invention will now be described, by way of example only, with reference to the following drawings, in which: [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram for outlining an ML coupling profile that uses resources efficiently to support efficient execution of ML applications, according to a preferred embodiment. [Diagram 2] 1 is a schematic diagram of a computer system having an artificial intelligence platform for supporting efficient execution of ML programs and effective allocation and use of computing resources, in accordance with a preferred embodiment; [Diagram 3] 3 is a block diagram illustrating an artificial intelligence platform and corresponding tools as shown in FIG. 2 and their associated application program interfaces, according to a preferred embodiment. [Figure 4] FIG. 1 is a flow diagram illustrating a process for ML modeling and elasticity of computational resource allocation in accordance with a preferred embodiment. [Diagram 5] FIG. 5 is a block diagram illustrating an example of a computer system / server of a cloud-based support system for implementing the systems and processes described above with respect to FIGS. 1-4, in accordance with a preferred embodiment. [Figure 6] FIG. 1 is a block diagram illustrating a cloud computing environment in accordance with a preferred embodiment. [Figure 7] 1 is a block diagram illustrating a set of functional abstraction model layers provided by a cloud computing environment in accordance with a preferred embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Features shown in the drawings are meant to illustrate some embodiments only and not all embodiments, unless expressly indicated otherwise.

[0015] It will be readily understood that the components of the present embodiments, as generally illustrated and described in the Figures herein, could be arranged and designed in a wide variety of configurations. Thus, the following detailed description of the present embodiments of apparatus, systems, methods and computer program products, as illustrated in the Figures, is not intended to limit the scope of the embodiments, as claimed, but merely represents selected embodiments.

[0016] Throughout this specification, references to "selected embodiments," "one embodiment," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment. Thus, the appearances of the phrases "selected embodiments," "in one embodiment," and "in one embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment.

[0017] The illustrated embodiments will be best understood by reference to the drawings, in which like parts are consistently designated by like numerals. The following description is intended by way of example only and merely illustrates certain selected embodiments of devices, systems and processes consistent with the claimed embodiments.

[0018] Machine learning follows a process of preparing data, training an algorithm, generating a machine learning model, and then making and refining predictions. Machine learning models generally fall into one of the basic categories: supervised machine learning, unsupervised machine learning, reinforcement machine learning, and deep learning. For example, deep learning employs neural networks as a model of the way the human brain processes information. The basic unit of a neural network is referred to as a neuron, which is typically organized into layers. Neural networks work by simulating a large number of interconnected processing units that have something in common with an abstract version of a neuron. Neural networks typically have three parts, including an input layer with units that represent the input field, one or more hidden layers, and an output layer with a single unit or multiple units that represent the target field. These units are connected with various connection strengths or weights. Input data is fed into the first layer, and values ​​are propagated from each neuron to all neurons in the next layer. Finally, the output layer delivers the results. Deep learning complex neural networks are designed to emulate how the human brain works, so that computers can be trained to support poorly defined abstractions and problems. Neural networks and deep learning are often used in image recognition, speech and computer vision applications. Machine learning can learn and generate predictions based on data. Machine learning can also use combined profiling data to gain insights into the effective allocation of resources at fine-grained intervals to affect efficient workload execution.

[0019] It is understood that ML models have extensive hardware requirements to adequately support the processing needs of the models. Examples of such hardware resource requirements include, but are not limited to, RAM, disk space, GPU, and CPU. Resource management may be allocated to support sharing of resources among multiple ML models. As shown and described herein, and as demonstrated in the figures, ML workload information is correlated with resource information to adequately allocate resources to support application execution. Resource usage on ML routines is monitored and collected during execution and then used to predict and allocate resources for subsequent ML workloads. Thus, ML techniques are utilized for resource allocation for subsequent ML workloads.

[0020] Referring to FIG. 1, a block diagram 100 is provided outlining an ML combined profile for supporting efficient execution of ML applications with effective use of resources. As shown, an ML execution model 110 is developed for ML application execution and resource usage. The ML execution model 110 uses machine learning techniques to learn, predict, and allocate resources to other ML applications. More specifically, the ML execution model 110 is a performance prediction model based on ML techniques. The execution model is constructed using historical data 120 from previous executions of ML applications and recently collected data 130 of previous ML execution model iterations. The functionality of the ML execution model 110 is to correlate application execution performance to appropriate computational resources. As shown herein, the ML execution model 110 interacts with a resource management system 140 to effectively and efficiently direct resources to one or more ML models 150 for application execution. Thus, as shown herein, the ML execution model 110 serves as a component for generating or collecting a combined profile of applications and resource management.

[0021] Referring to FIG. 2, a schematic diagram of a computer system 200 having an artificial intelligence platform for supporting efficient execution of ML programs and effective allocation and use of computational resources is shown. As shown, a server 210 is provided in communication with multiple computing devices 280, 282, 284, 286, 288, and 290 across a network connection 205. The server 210 is comprised of a processing unit 212, such as a processor, in communication with a memory 216 via a bus 214. The processor may be a CPU or a GPU. The server 210 is shown having an artificial intelligence (AI) platform 250 supporting collaboration to train machine learning models as a joint profile of application execution and resource usage. The computing devices 280, 282, 284, 286, 288, and 290 can communicate with each other and with other devices or components via one or more wired and / or wireless data communication links, where each communication link may include one or more wires, routers, switches, transmitters, receivers, etc. In this networked arrangement, the server 210 and network connections 205 enable communication detection, recognition and resolution. Other embodiments of the server 210 may be used with other components, systems, subsystems or devices, or combinations thereof, than those illustrated herein.

[0022] The AI ​​platform 250 is shown herein configured to receive input 202 from a variety of sources. For example, the AI ​​platform 250 may receive input from a network 205 and utilize data sources 270, also referred to herein as a corpus or knowledge base, to generate output or response content. The AI ​​platform 250 is provided with tools to support and enable machine learning collaboration and to elastically execute machine learning workloads using application-based profiling.

[0023] AI platform 250 functions as a platform for enabling and supporting collaboration among application execution and resource allocation and management. In one embodiment, AI platform 250 communicates the response output to members operatively coupled to server 210, such as one or more computing devices 280-290 across network 205.

[0024] The AI ​​platform 250 is shown herein to have several tools to support neural model collaboration, including a profile manager 252, a machine learning (ML) manager 254, a director 256, and a resource manager 258. The profile manager 252 functions to generate or collect a joint profile comprised of ML application execution and resource usage. The joint profile combines details of the ML application execution with resource usage data corresponding to the ML execution. Examples of details of the ML application execution include, but are not limited to, batch size, learning rate, number of iterations, model accuracy, etc. Examples of resources include, but are not limited to, quantity of CPUs utilized, CPU type, quantity of GPUs utilized, GPU type, etc. The profile manager 252 monitors or receives signals from the ML profile and collects resource usage by the corresponding ML model and ML routines that are part of the ML model in one embodiment. More specifically, it is understood in the art that the construction or training of an ML model and the corresponding ML routines within the ML model require the use of computational resources, including physical and virtual resources, to support the functionality of the routines and the corresponding processing. The profile manager 252 collects data related to these resources and associated resource consumption from the corresponding ML model and the ML routines that are part of the ML model. The collected data is used to predict resource allocations for subsequent ML application executions, i.e., whether the subsequent applications will have the same, increased or decreased, or a different set or configuration of parameters in comparison to the current workload. As illustrated herein, the profile manager 252 utilizes at least one callback, illustrated herein as callback function A 252A, to profile the application execution with one or more associated parameters and profile resource usage information.In one embodiment, the creation of a profile of resource usage information is an identification, and in one embodiment, an outline, of the computational resources used or required to support the execution of an application. Thus, profile manager 252 functions to capture the computational resources used and usage characteristics along with an application execution under its set of parameters.

[0025] As shown herein, the server 210 is operatively coupled to a data source 270. The data source 270 is shown herein with a library 272 of ML models derived from an ML application. Although only one library is shown, it is understood that the data source 270 may be comprised of multiple libraries, and the library 272 is for illustrative purposes. The library 272 is shown with ML models including ML model A 274A and ML model B 274B. The quantity of ML models is for illustrative purposes and should not be considered limiting. In one embodiment, the ML models are organized in the library based on subject matter or functionality. Each of the ML models is shown with an operatively coupled or associated profile, also referred to herein as a coupled profile. More specifically, ML model A 274A is shown with profile A 276A, and ML model B 274B is shown with profile B 276B. The ML models 274A and 274B in the library 272 are not necessarily coupled to one particular profile. An ML model may be built using multiple combined profiles. For example, as shown, ML model N274N is shown as a combination of profiles, such as a combination of profile C276C, profile D276D, and profile N276N. Each profile combines the execution of a corresponding or operatively combined model with resource usage or requirements to support the execution of the operatively combined model. For example, in one embodiment, ML models may be built with the same parameters but with different sets of computational resources. The resulting models will be the same, but the timing of when the models ultimately converge will be different due to resource constraints.

[0026] The ML manager 254 shown herein is operatively coupled to the profile manager 252 and functions to identify one or more features and signatures from the profiles, e.g., 276A and 276B. Examples of features and signatures are machine learning frameworks, training batch size, number of CPUs allocated, size of memory used, execution time for each iteration or algorithm convergence time. The ML manager 254 is shown herein with an operably coupled ML execution model 254A. More specifically, the model 254A is an ML model that uses ML techniques to predict computational resource usage for ML application execution. The ML manager 254 builds and manages the model 254A and executes the model 254A to assess and predict computational resource requirements for a corresponding ML model, e.g., 274A, by utilizing one or more features and signatures from the corresponding profile, e.g., 276A. In one embodiment, the identification of features and characteristics, e.g., signatures, for the generation of the combined profile and construction of the ML model 254A are derived using user input. Thus, ML manager 254 functions to create ML models that model the functionality and processing of other ML models, such as models 274A and 274B, to facilitate computational resource allocation to support ML model processing.

[0027] The director 256 functions to apply the ML execution model 254A and utilizes the output 254B from the model to provide instructions for subsequent application execution. The output 254B is in the form of computational resources necessary to support efficient and effective execution of one or more ML applications. The resource allocation may be elastic, e.g., dynamic, and may change during execution. The director 256 utilizes the output 254B based on the ML execution model 254A to elastically allocate and request one or more computational resources from the resource management component to support application execution. As shown herein, a resource manager 258 is shown operatively coupled to the director 256. In one embodiment, the output 254B is forwarded to the resource manager 258 with instructions indicating the computational resources and resource designation to support application execution. As described herein, resource allocation is defined as elastic, e.g., non-static. Similarly, the instructions corresponding to the output 254B are in the form of resource allocation actions. The ML execution model 254A executes resource allocation actions, e.g., directives, to request available resources to support application processing. The resource allocation actions may reduce or expand resources for previous application processing and execution, or, in one embodiment, the resource allocation actions may maintain the resource requirements. Similarly, the requested resources have corresponding attributes. The resource allocation actions identify resource attributes that support application processing and execution, which are elastic similar to resource allocation. Thus, the director 256 functions to support and process the output from the ML model 254A.

[0028] Iteration is a term used in ML applications to indicate the number of times an algorithm's parameters are updated. Training the corresponding neural network requires multiple iterations. With respect to the functionality described herein, computational resource allocation may change across iterations based on the requirements of the ML execution model and the availability of resources. The ML execution model 254A responds to the dynamic nature and iterations of the ML model, and the corresponding output 254B reflects this dynamic nature. The director 256 may change the allocation of computational resources across iterations of the ML application. In one embodiment, the resource allocation changes may be based on application execution patterns, resource usage patterns, or a combination of both. Any changes to the resource allocation are reflected in the output 254B and corresponding resource allocation actions, e.g., directives, across the ML model iterations.

[0029] In some exemplary embodiments, server 210 may be an IBM® Watson® system available from International Business Machines Corporation, Armonk, NY, and may be extended using mechanisms of the exemplary embodiments described below. The systems shown and described herein include tools for implementing machine learning based on iterative optimization algorithms. The tools enable elastic execution of ML workloads using application-based profiling, thereby effectively and efficiently allocating computational resources to ML applications and their corresponding neural networks, and across multiple iterations of model building or training.

[0030] Profile manager 252, ML manager 254, director 256 and resource manager 258 (collectively referred to herein as AI tools or AI platform tools) are shown as embodied or integrated within AI platform 250 of server 210. The AI ​​tools may be implemented in separate computing systems (e.g., 290) connected to server 210 over network 205. Where embodied, the AI ​​tools function to support and enable elasticity of ML application execution modeling and corresponding resource allocation in an iterative manner. Output 254A from ML execution model 254 may be in the form of executable instructions forwarded to resource manager 258 to indicate or specify computational resources to support execution by one or more models 274A and 274B.

[0031] The types of information handling systems that may utilize the AI ​​platform 250 range from small handheld devices such as handheld computer / cell phone 280 to large mainframe systems such as mainframe computer 282. Examples of handheld computers 280 include personal digital assistants (PDAs) and personal entertainment devices such as MP4 players, portable televisions, and compact disc players. Other examples of information handling systems include pen or tablet computers 284, laptop or notebook computers 286, personal computer systems 288, and servers 290. As shown, the various information handling systems may be networked together using computer network 205. Types of computer networks 205 that may be used to interconnect the various information handling systems include local area networks (LANs), wireless local area networks (WLANs), the Internet, public switched telephone networks (PSTNs), other wireless networks, or any other network topology that may be used to interconnect information handling systems. Many of the information handling systems include a non-volatile data store, such as a hard drive or non-volatile memory, or both. Some of the information handling systems may use a separate non-volatile data store (e.g., server 290 utilizes non-volatile data store 290A and mainframe computer 282 utilizes non-volatile data store 282A). Non-volatile data store 282A may be an external component of the various information handling systems or may be internal to one of the information handling systems.

[0032] The information handling system used to support the AI ​​platform 250 may take many forms, some of which are shown in Figure 2. For example, the information handling system may take the form of a computer or data processing system of a desktop, server, portable, laptop, notebook or other form factor. In addition, the information handling system may take other form factors, such as a personal digital assistant (PDA), gaming device, ATM machine, portable telephone device, communication device or other device that includes a processor and memory.

[0033] An application program interface (API) is understood in the art as a software intermediary between two or more applications. With respect to the AI ​​platform 250 shown and described in FIG. 2, one or more APIs may be utilized to support one or more of the tools 252-258 and associated functionality. With reference to FIG. 3, a block diagram 300 is provided illustrating the tools 252-258 and their associated APIs. As shown, multiple tools are embodied within the API platform 305, including a profile manager 352 associated with API0 312, an ML manager 354 associated with API1 322, a director 356 associated with API2 332, and a resource manager 358 associated with API3 342. Each of the APIs may be implemented in one or more languages ​​and interface specifications. API0312 provides functional support for generating a combined profile as a combination of application execution details and application execution resource usage, API1322 provides functional support for building and maintaining ML execution models targeted at characterizing and identifying ML application execution and computational resource usage, API2332 provides functional support for elastically requesting computational resources based on output from application and ML execution models, and API3342 provides functional support for directing and specifying computational resources to support execution by one or more of the models and their corresponding neural networks.

[0034] As shown, each of the APIs 312, 322, 332, and 342 is operatively coupled to an API orchestrator 360 (otherwise known as an orchestration layer), which is understood in the art to act as an abstraction layer that transparently threads the separate APIs together. In an embodiment, the functionality of the separate APIs may be combined or combined. Thus, the configuration of APIs shown herein should not be considered limiting. Thus, the functionality of the tools may be embodied or supported by their respective APIs, as shown herein. In another embodiment, the separate APIs may be processed in distributed computing resources to achieve elasticity, scalability, and availability.

[0035] Referring to FIG. 4, a flow diagram 400 is provided to illustrate the process for ML modeling and elasticity of computational resource allocation. As shown, a request to run an application, known herein as an execution request, is received (402) along with application-specific and system profiling-specific parameters. Execution instructions are generated (404) based on the received request from step 402 and based on the ML execution modeler 450, which entails application parameters and historical processing of the model. In one embodiment, if historical execution is unknown or unavailable, only user-specified input may be utilized. Possible execution instructions include, but are not limited to, setting limits on memory usage and allocating resources for usage. With respect to memory usage, limits may be set for an instance of an application based on historical data from one or more previous iterations or one or more previous executions. With respect to resource utilization, the prior historical data may include information regarding resource utilization for the same tuning parameters of the training model, but on different computing platforms, e.g., different quantities of CPUs, CPU types, different quantities of GPUs, GPU types, and NVLink (a wire-based communication protocol) connectivity. These instructions are for illustrative purposes and should not be considered limiting. Thus, the instructions are generated based on parameters corresponding to the ML execution model and execution requirements.

[0036] The directives generated in step 404 are used to interact with resource management to obtain desired resources to support the execution request and to perform resource allocation within the request parameters (406). More specifically, the directives obtained in step 404 are used to allocate computational resources. For example, in one embodiment, the allocation may be directed to containers of a specified size for elastic execution for potential dynamic expansion to the number of CPU cores, CPU memory, GPU type, and number of GPUs. Similarly, in one embodiment, the allocation may be directed to packaging containers on the same machine if possible or desirable. In another embodiment, the allocation may be directed to GPU sharing. After the resources are allocated, one or more ML applications are executed (408) and profile information of the executed ML applications is collected (410). In one embodiment, the collection uses callback functions for details of the ML application and resource usage information. The collected ML application execution data is referred to herein as dynamic application information and is communicated to the ML execution modeler 450. Following step 410 and the profile information collection, a determination is made as to whether a termination threshold has been reached 412. A negative response to the determination at step 412 follows a return to step 404, and an affirmative response causes the profile collection and deployment process to terminate.

[0037] The ML execution modeler 450 receives inputs from application execution and interaction with resource management. The ML execution modeler 450 receives both static input data 460 and dynamic input data 470. The static input data is in the form of input data information 462, tuning parameter ranges 464, and static computation information such as CPU, GPU architecture, memory and storage 466. The dynamic input data 470 is in the form of parameter settings 472 and resource usage information 474. Thus, the ML execution modeler 450 utilizes the application execution information and resource information to identify optimal resources and build predictions of requirements to support the ML application.

[0038] As shown in FIG. 4, the callback function is used by the executed or running ML application to identify ML application data and ML application resource usage, thereby generating a combined profile of ML workload and resource usage. In one embodiment, the combined profile may include time-stamped data having ML application specific features or signatures and resource usage. The ML execution model built in the ML execution modeler utilizes the generated combined profile that encompasses the application execution details. More specifically, the ML execution model is dynamic in that it receives and applies updates from the combined profile. The ML execution model is built using data from previous runs or is built from recently collected data from previous iterations in the same run. The features and signatures included in the combined profile and the updates dynamically applied in the combined profile provide instructions for the application execution of subsequent iterations and correlate the application execution performance with appropriate, and in one embodiment, heterogeneous, computational resources. In one embodiment, a set of APIs is employed to support callback functions from the executed or running ML application. The set of APIs is callable from applications in any ML framework. Thus, the combined profile is both used to construct the ML execution model in the ML execution modeler, and once the combined profile is constructed, it is applied by the ML execution modeler to effectively request resources from the resource management component to efficiently execute the ML application at various iterations.

[0039] Below is an example of trace output from an API to capture details about how an application is running.

number

[0040] Aspects of the functional tools 252-258 and their associated functionality may be embodied in a computer system / server at a single location or, in one embodiment, configured in a cloud-based system sharing computing resources. Referring to Figure 5, a block diagram 500 illustrates an example of a computer system / server 502, referred to as a host 502, in communication with a cloud-based support system for implementing the processes described above in connection with Figures 1-4. The host 502 is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments or configurations, or combinations thereof, suitable for use with host 502 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and file systems that include any of the above systems, devices, and equivalents thereof (e.g., distributed storage environments and distributed cloud computing environments).

[0041] The host 502 may be described in the general context of computer system executable instructions, such as program modules, executed by a computer system. Generally, program modules include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The host 502 may be implemented in a distributed cloud computing environment 580, where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

[0042] As shown in FIG. 5, the host 502 is shown in the form of a general-purpose computing device. The components of the host 502 may include a processor or processing unit 504, such as, but not limited to, a hardware processor, a system memory 506, and a bus 508 that couples various system components, including the system memory 506, to the processor 504. The bus 508 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. The host 502 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by the host 502 and includes both volatile and non-volatile media, removable and non-removable media.

[0043] Memory 506 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 530 or cache memory 532, or both. By way of example only, a storage system 534 may be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown, but commonly referred to as a "hard drive"). Not shown, a magnetic disk drive may be provided for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive may be provided for reading from and writing to a removable, non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical medium. In such an example, each may be connected to bus 508 by one or more data media interfaces.

[0044] A program / utility 540 having a set (at least one) of program modules 542 may be stored in the memory 506, for example and not by way of limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or any combination thereof, may include an implementation in a network environment. The program modules 542 generally perform the functions and / or methodologies of an embodiment of dynamic communication evaluation interrogatory identification and processing. For example, the set of program modules 542 may include tools 252-258 illustrated in FIG. 1.

[0045] The host 502 may communicate with external devices 514, such as a keyboard, pointing device, etc.; a display 524; one or more devices that allow a user to interact with the host 502; or any device (e.g., network card, modem, etc.) that allows the host 502 to communicate with one or more other computing devices, or a combination thereof. Such communication may occur via an input / output (I / O) interface 522. Additionally, the host 502 may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, via a network adapter 520. As shown, the network adapter 520 communicates with other components of the host 502 via a bus 508. In one embodiment, multiple nodes of a distributed file system (not shown) may communicate with the host 502 via the I / O interface 522 or via the network adapter 520. Although not shown, it is understood that other hardware and / or software components may be used with the host 502. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives and data archive storage systems.

[0046] In this document, the terms "computer program medium," "computer usable medium," and "computer readable medium" are used generally to refer to media such as main memory 506, including RAM 530, cache 532, and storage system 534, such as a removable storage drive and a hard disk installed in a hard disk drive.

[0047] Computer programs (also referred to as computer control logic) are stored in memory 506. Computer programs may also be received via a communications interface, such as network adapter 520. Such computer programs, when executed, enable the computer system to perform the functions of the present embodiments as described herein. In particular, the computer programs, when executed, enable processing unit 504 to perform the functions of the computer system. Thus, such computer programs represent controllers of the computer system.

[0048] A computer readable storage medium may be a tangible device that holds and stores instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, dynamic or static random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), magnetic storage devices, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, punch cards or mechanically encoded devices such as ridge structures in grooves having recorded instructions, and any suitable combination of the above. Computer-readable storage media, as used herein, is not to be construed as a transitory signal per se, such as an electric wave, a freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0049] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage in the computer readable storage medium within the respective computing / processing device.

[0050] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object oriented languages ​​such as Java, Smalltalk, C++ or the like, conventional procedural languages ​​such as the C programming language or similar programming languages. The computer readable program instructions may be executed as a stand-alone software package, entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server or cluster of servers. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, the electrical circuitry may include, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), and the state information of the computer readable program instructions may be utilized to individualize the electrical circuitry to execute the computer readable program instructions to perform aspects of the present embodiments.

[0051] In one embodiment, host 502 is a node of a cloud computing environment. As known in the art, cloud computing is a model of service delivery that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models. Examples of these characteristics are: On-Demand Self-Service: Cloud consumers can unilaterally provision computing power, such as server time and network storage, as they need it automatically without the need for human interaction with the service provider. Broadband Network Access: Capabilities are available over the network and accessed via standard mechanisms that facilitate use by heterogeneous thin-client or thick-client platforms (e.g., mobile phones, laptops, PDAs). Resource Pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where various physical and virtual resources are dynamically allocated and reallocated according to demand. Consumers generally have no control or knowledge of the exact location of the resources provided to them, but there is a sense of location independence in the sense that they can specify location at a higher level of abstraction (e.g. country, state, or data center). Rapid Elasticity: Capacity can be rapidly and elastically, in some cases automatically, provisioned to quickly scale out and rapidly released to quickly scale in. To the consumer, the provisioned available capacity often appears externally unlimited, and any amount can be purchased at any time. Measured Services: Cloud systems automatically control and optimize resource usage by utilizing metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, number of active users). Resource usage is monitored, controlled, and reported, providing transparency to both providers and consumers of the services being used.

[0052] The service model is as follows: Software as a Service (SaaS): The capability offered to the consumer is to use a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities with the potential exception of limited user-specific application configuration settings. Platform as a Service (PaaS): The capability offered to a consumer is to deploy consumer-created or acquired applications, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does have control over the deployed applications and possibly the configuration of the application hosting environment. Infrastructure as a Service (IaaS): The capability provided to the consumer is to provide processing, storage, networking, and other basic computational resources onto which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does have control over the operating systems, storage, deployed applications, and in some cases, limited control over selected networking components (e.g., host firewalls).

[0053] The deployment model is as follows: Private Cloud: The cloud infrastructure is used exclusively for one organization. It may be managed by the organization or a third party and may be on-premise or off-premise. Community Cloud: Cloud infrastructure is shared by several organizations to support a specific community with common concerns (e.g., mission, security requirements, policy and compliance considerations). It may be managed by the organization or a third party and may exist on-premise or off-premise. Public Cloud: The cloud infrastructure is available to the general public or large industry entities and is owned by organizations that sell cloud services. Hybrid Cloud: A cloud infrastructure is a blend of two or more clouds (private, community or public) that remain unique entities but are joined by standardized or proprietary technologies that allow data and application portability (e.g. cloud bursting for load balancing between clouds). Cloud computing environments are service-oriented with an emphasis on statelessness, low coupling, modularity and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0054] Referring now to FIG. 6, an exemplary cloud computing network 600 is shown. As shown, the cloud computing network 600 includes a cloud computing environment 650 having one or more cloud computing nodes 610 with which local computing devices used by, for example, cloud consumers may communicate. Examples of these local computing devices include, but are not limited to, a personal digital assistant (PDA) or mobile phone 654A, a desktop computer 654B, a laptop computer 654C, or an automobile computer system 654N, or combinations thereof. Individual nodes within the nodes 610 may further communicate with each other. These may be physically or virtually grouped (not shown) in one or more networks, such as private, community, public, or hybrid clouds, as described above, or combinations thereof. This enables the cloud computing environment 600 to provide infrastructure, platform, or software, or combinations thereof, as a service, for which the cloud consumer does not need to maintain resources on the local computing device. The types of computing devices 654A-654N shown in FIG. 6 are for illustrative purposes only, and it will be understood that the cloud computing environment 650 can communicate with any type of computerized device via any type of network, network-addressable connection (e.g., using a web browser), or both.

[0055] Now, referring to Fig. 7, a set of functional abstraction layers 700 provided by the cloud computing network of Fig. 6 is shown. It should be understood in advance that the components, layers and functions shown in Fig. 7 are for illustrative purposes only, and the present embodiment is not limited thereto. As shown, the following layers and corresponding functions are provided: a hardware and software layer 710, a virtualization layer 720, a management layer 730 and a workload layer 740.

[0056] The hardware and software layer 710 includes hardware and software components. Examples of hardware components include mainframes, in one example, the IBM® zSeries® System, servers based on RISC (Reduced Instruction Set Computer) architecture, in one example, the IBM® pSeries® System, the IBM® xSeries® System, the IBM® BladeCenter® System, storage devices, and networks and networking components. Examples of software components include network application server software, in one example, the IBM® WebSphere® application server software, and database software, in one example, the IBM® DB2® database software (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere, and DB2 are registered trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide).

[0057] The virtualization layer 720 provides an abstraction layer from which instances of virtualized entities such as virtualized servers, virtualized storage, virtualized networks including virtual private networks, virtualized applications and operating systems, and virtual clients are provided.

[0058] In one example, the management layer 730 may provide the functionality of resource provisioning, metering and pricing, user portal, service level management, and SLA planning and fulfillment. Resource provisioning provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing provides tracking of costs at which resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. The user portal provides access to the cloud computing environment for consumers and system administrators. Service level management provides allocation and management of cloud computing resources to meet required service layers. Service level agreement (SLA) planning and fulfillment provides pre-allocation and procurement of cloud computing resources in anticipation of future demands according to SLAs.

[0059] The Workloads tier 740 provides examples of functionality for which cloud computing environments are utilized. Examples of workloads and functions provided from this tier include, but are not limited to, mapping and navigation, software development and lifecycle management, virtual classroom instructional delivery, data analytics processing, transaction processing, and ML execution modeling.

[0060] It is understood that disclosed herein are systems, methods, apparatus, and computer program products for evaluating natural language input, detecting questions in corresponding communications, and resolving the detected questions with responses or supporting content, or both.

[0061] Although specific embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that, based on the teachings herein, changes and modifications can be made without departing from the embodiments and their broader aspects. Accordingly, the appended claims are intended to encompass within their scope all such changes and modifications as are within the true spirit and scope of the present invention. Moreover, it should be understood that these embodiments are defined solely by the claims. Where a particular number of introduced claim elements is intended, such intent will be expressly stated in the claims, and in the absence of such a statement, those skilled in the art will understand that there is no such limitation. For non-limiting examples, and to aid in understanding, the following claims include the use of the phrases "at least one" and "one or more" to introduce claim elements. However, the use of such phrases, even in the same claim containing the phrase "one or more" or "at least one" and an infinitive such as "a" or "an," should not be construed as implying that the introduction of a claim element with the infinitive "a" or "an" limits such element to embodiments containing only one of the element, and this also applies to the use of definite articles in the claims.

[0062] The present embodiments may be systems, methods or computer program products, or combinations thereof. In addition, selected aspects of the present embodiments may take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining aspects of software or hardware or both, all generally referred to herein as "circuits," "modules," or "systems." Furthermore, aspects of the present embodiments may take the form of a computer program product embodied in a computer-readable storage medium (or media) having computer-readable program instructions thereon to cause a processor to execute aspects of the present embodiments. Thus, the disclosed systems, methods, or computer program products, or combinations thereof, operate to improve the functionality and operation of artificial intelligence platforms for modeling ML application execution and resource usage, including efficient and effective ML application performance and resource management.

[0063] A computer readable storage medium may be a tangible device that holds and stores instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, dynamic or static random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), magnetic storage devices, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, punch cards or mechanically encoded devices such as ridge structures in grooves having recorded instructions, and any suitable combination of the above. Computer-readable storage media, as used herein, is not to be construed as a transitory signal per se, such as an electric wave, a freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0064] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage in the computer readable storage medium within the respective computing / processing device.

[0065] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object oriented languages ​​such as Java, Smalltalk, C++ or the like, conventional procedural languages ​​such as the C programming language or functional programming languages ​​such as Haskell, Lisp, and similar programming languages. The computer readable program instructions may be executed as a stand-alone software package, entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server or cluster of servers. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, the electrical circuitry may include, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), which may execute the computer readable program instructions by individualizing the electrical circuitry using state information of the computer readable program instructions to perform aspects of the embodiments.

[0066] Aspects of the present embodiments are described herein with reference to flowchart and / or block diagrams of methods, apparatus (systems) and computer program products according to the embodiments. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, may be implemented by computer readable program instructions.

[0067] These computer readable program instructions may be provided to a general purpose computer, special purpose computer processor or other programmable data processing apparatus to generate a machine such that the instructions, executed via the computer processor or other programmable data processing apparatus, create means for implementing the functions / acts identified in a block or blocks of the flowchart illustrations and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium capable of directing a computer, programmable data processing apparatus or other device or combination thereof to function in a particular manner, such that a computer readable storage medium having instructions stored thereon includes an article of manufacture including instructions that implement aspects of the functions / acts identified in a block or blocks of the flowchart illustrations and / or block diagrams.

[0068] The computer readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device and cause the computer, other programmable data processing apparatus, or other device to perform a series of operational steps to generate a computer-implemented process such that the instructions executing on the computer, other programmable data processing apparatus, or other device implement an aspect of the function / action identified in a block or blocks of the flowchart illustrations and / or block diagrams.

[0069] The flowchart diagrams and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart diagrams or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a particular logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order depicted in the drawings. For example, two blocks shown in succession may in fact be executed simultaneously, substantially simultaneously, or the blocks may be executed in reverse order depending on the functionality involved. It should be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a special purpose hardware-based system that performs a particular function or function or implements a combination of special purpose hardware and computer instructions.

[0070] Although specific embodiments have been described herein for purposes of illustration, it will be understood that various modifications may be made without departing from the spirit and scope of the embodiments. Accordingly, the scope of protection of the embodiments is limited only by the following claims and their equivalents.

Claims

1. 1. A computer system comprising: A processing unit; a memory operatively coupled to the processing unit; the memory comprising: generating a profile for one or more machine learning (ML) applications comprising ML application execution data and resource usage data; identifying one or more features and one or more signatures from the generated profile; constructing an ML execution model configured to correlate ML application execution performance to one or more computational resources, the ML execution model utilizing the identified one or more features and one or more signatures; applying, by the ML execution model, one or more instructions to subsequent application execution to dynamically allocate and request one or more resources from a resource management component to at least one of the one or more ML applications to support application execution, the ML execution model being configured to receive and apply updates from the generated profile and change allocations during execution of the one or more ML applications; and storing instructions to cause the processing unit to execute the steps of:

2. The computer system of claim 1, wherein the ML execution model is configured to use one or more ML techniques to evaluate one or more resource requirements for at least one of the one or more ML applications and generate an output to identify one or more computational resources for supporting execution of the one or more ML applications, and the one or more instructions are consistent with the generated output.

3. said dynamically allocating includes said ML execution model performing resource allocation actions with respect to resource availability and application processing support; 3. The computer system of claim 1, wherein the resource allocation action is to reduce or expand any resource, and wherein one or more of the reduced or expanded resources have the same or different attributes.

4. The computer system of any one of claims 1 to 3, wherein generating the profile comprises monitoring and collecting resource usage data for one or more ML routines and using the monitored resource usage data to predict allocations for future ML applications.

5. The computer system of any one of claims 1 to 4, wherein the identification of the one or more features and one or more signatures for generating the profile and for building the execution model are derived from user input.

6. A computer system comprising: a processing unit operatively coupled to the memory; an artificial intelligence (AI) platform in communication with the processing unit, the AI ​​platform having tools for dynamically executing one or more machine learning workloads with application-based profiling; said tool comprising: a profile manager that generates a combined profile comprising machine learning (ML) application execution data and resource usage data; a machine learning manager that identifies one or more features and one or more signatures from the generated combined profile and builds an ML execution model for ML application execution performance and resource usage, the ML execution model utilizing the identified one or more features and one or more signatures; a director that applies the ML execution model, the ML execution model providing one or more directives to subsequent application execution, including dynamically allocating and requesting one or more resources from a resource management component to support application execution; wherein dynamically allocating includes the ML execution model performing resource allocation actions with respect to resource availability and application processing support; The resource allocation action is for reducing or expanding any resource, one or more of the reduced or expanded resources having the same or different attributes, and the computer system further comprises: A computer system comprising: said director for varying allocation of computational resources across one or more iterations of a ML application, including invoking said resource allocation actions based on application execution patterns and resource usage patterns.

7. A computer system comprising: a processing unit operatively coupled to the memory; an artificial intelligence (AI) platform in communication with the processing unit, the AI ​​platform having tools for dynamically executing one or more machine learning workloads with application-based profiling; said tool comprising: a profile manager that generates a combined profile comprising machine learning (ML) application execution data and resource usage data; a machine learning manager that identifies one or more features and one or more signatures from the generated combined profile and builds an ML execution model for ML application execution performance and resource usage, the ML execution model utilizing the identified one or more features and one or more signatures; a director that applies the ML execution model, the ML execution model providing one or more directives to subsequent application execution, including dynamically allocating and requesting one or more resources from a resource management component to support application execution; wherein the profile manager utilizes one or more callback functions to profile application execution with one or more application-related parameters and profile resource usage information.

8. Generating a profile for one or more machine learning (ML) applications, the profile comprising ML application execution data and resource usage data; identifying one or more features and one or more signatures from the generated profile; constructing an ML execution model configured to correlate ML application execution performance to one or more computational resources, the ML execution model utilizing the identified one or more features and one or more signatures; applying one or more instructions to subsequent application execution to dynamically allocate and request one or more resources from a resource management component to at least one of the one or more ML applications to support application execution by the ML execution model, the ML execution model being configured to receive and apply updates from the generated profile and change allocations during execution of the one or more ML applications; A method comprising:

9. The method of claim 8, wherein the ML execution model is configured to use one or more ML techniques to evaluate one or more resource requirements for at least one of the one or more ML applications and generate an output to identify one or more computational resources for supporting execution of the one or more ML applications, and the one or more instructions are consistent with the generated output.

10. The dynamically allocating step comprising: performing resource allocation actions with respect to resource availability and application processing support; The method of claim 8 or 9, wherein the resource allocation action is to reduce or expand any resource, the reduced or expanded resource(s) having the same or different attributes.

11. The method of any one of claims 8 to 10, wherein generating the profile includes monitoring and collecting resource usage data for one or more ML routines, and using the monitored resource usage data to predict allocations for future ML applications.

12. The method of any one of claims 8 to 11, wherein the identification of the one or more features and one or more signatures for generating the profile and for building a performance model are derived from user input.

13. Generating a combined profile comprising machine learning (ML) application execution data and resource usage data; identifying one or more features and one or more signatures from the generated combined profile and constructing an ML execution model for ML application execution performance and resource usage, the ML execution model utilizing the identified one or more features and one or more signatures; applying said ML execution model to provide one or more instructions to subsequent application execution, including dynamically allocating and requesting one or more resources from a resource management component to support application execution; The method includes: said dynamically allocating including performing resource allocation actions with respect to resource availability and application processing support; The resource allocation action is to reduce or expand any resource, one or more of the reduced or expanded resources having the same or different attributes, and the method includes: Varying computational resource allocations across one or more iterations of a ML application, including invoking the resource allocation actions based on application execution patterns and resource usage patterns. The method further comprising:

14. Generating a combined profile comprising machine learning (ML) application execution data and resource usage data; identifying one or more features and one or more signatures from the generated combined profile and constructing an ML execution model for ML application execution performance and resource usage, the ML execution model utilizing the identified one or more features and one or more signatures; applying said ML execution model to provide one or more instructions to subsequent application execution, including dynamically allocating and requesting one or more resources from a resource management component to support application execution; wherein generating the combined profile utilizes one or more callback functions to profile application execution with one or more application-related parameters and profile resource usage information.

15. A computer program comprising program code means adapted to perform the method according to any one of claims 8 to 14, when the program is run on a computer.

16. A recording medium having stored thereon a computer program for supporting dynamic execution of machine learning workloads using application-based profiling, the computer program being configured to cause a processor to execute a method according to any one of claims 8 to 14.

Images