System for Software Module Development

Abstract

Systems and methods for software module development. A configuration file and a process agent module cooperate with a computer system to pre-configure one or more execution environments to implement one or more instances of a user software module under development. The configuration file contains hardware and software configurations that define the limitations and capabilities of the execution environment and the parameters required by the software module. The process agent starts the execution environment and ensures that the software module executed in the execution environment can access the resources set in the configuration file. Once the execution of the software module is complete, the performance results are passed to the process agent for verification and analysis. These results can then be used to determine which implementation of the software module performed best.

Description

[0001] Cross - Reference to Related Applications

[0002] This application is a continuation of PCT application PCT / CA2020 / 050462, filed on April 8, 2020, which claims the benefit of U.S. Provisional Application 62 / 833,192, filed on April 12, 2019, both of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present invention relates to software development. More specifically, the present invention relates to systems and methods for developing software modules. Background Art

[0004] In the past few years, there has been a surge in the interest and development of artificial intelligence, more specifically machine learning and neural networks, resulting in a rapid increase in the demand for systems for developing such technologies. To this end, more and more computer scientists, developers, and engineers are pushing the technological boundaries to obtain better and better systems. However, this also poses many potential problems for these scientists, engineers, and developers.

[0005] When developing software, different configurations can produce different results in terms of performance. Therefore, in order to optimize the performance of the software being developed, developers need to keep track of these different configurations and ensure that the configuration that provides the best results is maintained. However, some software may have dozens or even hundreds of configurations, and keeping track of so many parameters and settings is burdensome even in the best of cases. Similarly, the configuration of the system on which the software runs also affects the performance of the software. Therefore, it is also necessary to keep track of the system configuration to ensure that all configurations of the software are competing on a level playing field. Similarly, this can become a burdensome task.

[0006] It should be clear that all of the above becomes even more challenging since various forms of execution environments can be used to execute software development. To ensure that all configurations of the software being developed are correctly evaluated, the configurations of the execution environments in which such software is executed will have to be as identical as possible to each other.

[0007] When developing neural network and machine learning systems, the above problems are even more acute. The hyperparameters used by such systems can easily reach hundreds, and ensuring that each version of the software system being developed runs in the same execution environment can be difficult even in the best of cases. Moreover, each change in the hyperparameters used can produce different results, and therefore, it is necessary to collate, track, and manage the performance of each system to ensure that the correct parameter settings are associated with the correct performance metrics.

[0008] Therefore, a system and / or method for solving the above problems is needed. Preferably, such a method or system can alleviate, if not overcome, the above problems, and also preferably, such a system or method is also easy for developers and / or researchers to use. Summary of the Invention

[0009] The present invention provides a system and method for software module development. The configuration file and the process agent module cooperate with the computer system to pre - configure (provision) one or more execution environments to implement one or more instances of the user software module under development. The configuration file contains the hardware and software configurations that define the limitations and capabilities of the execution environment and the parameters required by the software module. The process agent starts the execution environment and ensures that the software module executed in the execution environment can access the resources set in the configuration file. Once the execution of the software module is completed, the performance results are passed to the process agent for verification and analysis. Then these results can be used to determine which implementation of the software module performs the best. In a specific implementation, the process agent itself is a virtual machine and manages the configuration, pre - configuration, and startup of other virtual machines / execution environments and other jobs.

[0010] In a first aspect, the present invention provides a system for pre - configuring and starting one or more instances of a software module, the system comprising:

[0011] - A configuration file that details the hardware and software configurations for implementing at least one instance of the software module;

[0012] - A process agent module for configuring at least one computer system to implement the at least one instance of the software module, the process agent using the configuration file to configure the at least one computer system;

[0013] wherein

[0014] - The configuration file is stored in a data storage device such that the file is retrievable and such that each implementation of the software module uses the configuration file;

[0015] - The configuration file is uniquely identified by an identifier that is uniquely associated with the software module.

[0016] In a second aspect, the present invention provides a system for starting multiple instances of a software module, the system comprising:

[0017] - A configuration file that details the configurations for implementing the multiple instances of the software module;

[0018] - A process agent module, which is used to configure at least one computer system to implement the multiple instances of the software module, and the process agent uses the configuration file to configure the at least one computer system;

[0019] wherein

[0020] - The configuration file is uniquely identified by an identifier, and the identifier is uniquely associated with the software module. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Embodiments of the present invention will now be described with reference to the following drawings, where like reference numerals in different drawings represent the same elements and wherein:

[0022] Figure 1 is a block diagram showing components in a system according to one aspect of the present invention. DETAILED DESCRIPTION

[0023] In one aspect, the present invention provides a system including a configuration file and a process agent module. These two cooperate to solve the above problems. In operation, the configuration file is prepared by the user and is uniquely associated with the software module that the user is preparing / developing. The configuration file is prepared to include the software and hardware configurations required for the virtual machine that will run the user software module. This may include: how much RAM (random access memory) to provide for the virtual machine; how many processor cores (or multiple processors) to allocate / provide for the virtual machine; how many GPUs are dedicated to the virtual machine; and, in some implementations using a multitasking environment, what priority to give to the processes of the software module. In addition to this, the configuration file can also include ranges of various parameters that the software module may require. For example, the software module can be an implementation of a neural network, and in this case, each node in the neural network will require hyperparameter values. The configuration file can include ranges of various hyperparameters for each node in the neural network.

[0024] In cooperation with the configuration file, the process agent is used to configure the execution environment based on the content of the configuration file. Therefore, the process agent reads the content of the configuration file and pre-configures one or more execution environments based on this content when necessary. In addition, the process agent actually starts the software module that is uniquely associated with the configuration file and, when necessary, uses the parameter values in the configuration file again for the relevant components of the software module. It should be clear that the configuration file can also contain parameters such as how many trials / tests / implementations of the software module are to be executed. Therefore, the process agent adopts these trial parameters and pre-configures the necessary number of execution environments to run the necessary number of software module instances in parallel. For example, if the configuration file indicates that five instances of the software module are required, each with a specific set of parameters to be used, then the process agent will configure and start five identical execution environments, and each environment will execute the software module with the relevant parameters.

[0025] In addition to the above, the process agent can also receive and verify any results generated by various instances of the software module being tested and / or developed. This includes performance data for various configured instances (via parameters) of the software module. By doing so, the process agent can collect, verify, and rank (by performance results) the various results. Thus, the process agent can determine which parameters to use to generate the best performance configuration for the software module. These results and rankings can then be presented to the user, and if desired or required, the user can rewrite / modify the configuration file such that the optimized parameters are in the configuration file.

[0026] For clarity, the concept of an execution environment controlled, preconfigured, and launched by a configuration file and a process agent includes individual resources (such as CPUs, RAM, GPUs, network adapters) executing on a platform-independent environment that is capable of exposing these resources to a set of user-defined programs. The term "execution environment" includes examples such as virtual machines, process virtual machines, docker containers, and full virtualization or emulation of physical machines. Thus, when the process agent launches an execution environment, this may include launching a specially preconfigured virtual machine, emulation of a physical machine (preconfigured with specific resources), or simply launching a specific software environment (i.e., software module) preconfigured with specific resources available to user programs.

[0027] It should be clear that the use of a configuration file and a process agent also provides additional advantages. Once the best (or near-optimal) parameters for a software module have been found and entered into the configuration file, the file can be stored in a data storage device. Whenever the software module must be executed or implemented again, the process agent can simply retrieve the configuration file from the data storage device and use exactly the same hardware and software configuration as the one that was found to be most suitable for the software module. Similarly, the parameters that have been found to provide the best performance for the software module (now part of the configuration file) are also used. This ensures that the best configuration is used when implementing the software module. This also ensures that, if a researcher or developer needs to rerun the software module, the version used is the one that produced the best performance results. Additionally, this ensures that the conditions surrounding the hardware and software configuration of the software module are the same as when the software module was previously implemented. Researchers / developers no longer need to carefully document which configuration was used for which experiment / version because all versions of the software module will run on the same hardware and software configuration. However, the configuration of the software module does not have to be based solely on the past execution history of the software module. The configuration and parameters in the configuration file can also be based on the specific needs of the software module and any models that the software module can implement.

[0028] It should be noted that, for ease of identifying which configuration file is associated with which software module, the configuration file can be uniquely associated with the software module. This can be accomplished through unique identification codes embedded in the configuration file and the software module. Thus, before implementing the software module, the process agent can examine the software module and search the data storage device to determine whether there is a configuration file with the same identification code. If such a configuration file exists, the configuration file is retrieved and then implemented or used with its configuration and parameters together with the execution environment that will run the software module.

[0029] To ensure that the system of the present invention is easy to use, the process agent and the configuration file should be independent of the characteristics of the software module. This means that TensorFlow, Python, Torch, or any other suitable system and / or programming language can be used to develop / create the software module. Preferably, the configuration file will contain an indication of the type / style of the system used to create the software module. Then the process agent can use this indication to ensure that the parameters in the configuration file are applicable to that particular system. Similarly, the process agent can use this indication to ensure that the necessary libraries and support resources of the system are available for the software module. In an alternative, the configuration file can also contain an indication of libraries or other modules (i.e., libraries or other modules that are not part of the standard set) that the software module may need for execution / implementation. Then the process agent can use it to ensure that these libraries or modules are available for and accessible by the various virtual machines launched by the process agent. It should also be clear that, in one implementation, the process agent is a virtual machine that launches other virtual machines (or jobs or pre-configures other forms of execution environment) based on the functionality of the system as a whole, as described herein. Alternatively, the process agent can be an independent subsystem that launches virtual machines or pre-configures the execution environment. As described herein, the process agent (whether as a virtual machine or an independent subsystem or process) supervises the configuration, pre-configuration, and execution / launch of the execution environment based on the content in the configuration file. The process agent can reside in / initiate in one server / cluster to launch / control virtual machines on other servers / clusters, or the process agent and the execution environment it launches may both reside on the same server / cluster.

[0030] For an implementation specific to a neural network-based software module, the configuration file can include a range of values used by the neural network as hyperparameters. These values can then be used as a basis for the process agent to launch multiple instances of the software module (i.e., multiple instances of the neural network, each with a different set of hyperparameters based on the given range in the configuration file). In doing so, the process agent thus causes multiple parallel instances of the software module to execute on the same virtual machine / execution environment. Although these multiple instances of the software module can execute in parallel simultaneously, depending on the implementation of the system, the process agent can configure the various virtual machines / execution environments to operate sequentially or in an interleaved parallel or simultaneous parallel manner.

[0031] In a variant of the present invention, the system can determine the optimal configuration of one or more software modules based on the execution history of the one or more modules, regardless of what is in the configuration files of these modules. In one implementation, if a particular module has been executed enough times (the threshold is a configurable variable), the system provides the data of these executions to a machine learning model to determine the optimal values of one or more configuration parameters. Then, those one or more optimal parameters will be used to pre-configure or configure subsequent executions of the particular module. For example, after 100 executions of a software module, the configuration file lists 5 GPUs pre-configured for the process, and the execution data is sent to the machine learning system. If the machine learning system then determines (based on the execution data) that although 5 GPUs have been pre-configured, the particular module only actually uses 3 of them, then subsequent executions of the particular module will be configured with 3 GPUs instead of the 5 GPUs listed in the configuration file. Thus, the system can predictively determine the resource consumption of the software module based on a sufficient amount of execution data of the software module. Then the predicted consumption can be used to pre-configure one or more future executions of the software module, with the aim of optimizing or reducing the resource consumption of the module. The system can also determine what the highest minimum amount of resources the execution will require, so that the software module does not run out of resources.

[0032] Referring to Figure 1 , a block diagram of the components of a system according to one aspect of the present invention is shown. System 10 includes a configuration file 20 and a process agent 30. The process agent 30 operates in conjunction with a computer system 40 to pre-configure, configure, and start one or more virtual machines / execution environments 50 based on the configuration parameters listed in the configuration file 20. After various execution environments have been started, the configuration file 20 can be stored in a data storage device 60 for later use. When it is necessary to re-run or re-implement a software module, the process agent 30 can retrieve the configuration file 20 from the data storage device 60.

[0033] As described above, the performance metrics of the started software module can be sent to the process agent for collection, verification, and analysis. Once the optimal operating parameters of the software module have been determined, whether automatically through the analysis of the performance metrics by the process agent or manually through user analysis and selection of the parameters used by the software module, these parameters can then be included in the configuration file stored in the data storage device. This ensures that any future execution or implementation of the software module will use the optimal parameters now stored in the configuration file.

[0034] In summary, the system can also automatically determine the preferred optimal settings and parameters based on the execution data of the software modules collected. Using a machine learning model or any other suitable data analysis model, the system analyzes a suitable number of execution data sets of the software modules, and based on the results of this analysis, the system can predict the optimal parameters and settings of the modules. These parameters and settings can then be used as needed to replace the parameters in the configuration file. As described above, this capability allows researchers or users to ensure the optimization of future executions of the software modules and to ensure the use of the least amount of resources while ensuring that the software modules do not exhaust resources.

[0035] It should be clear that various aspects of the present invention can be implemented as software modules in an entire software system. Thus, the present invention can take the form of computer-executable instructions that, when executed, implement various software modules with predefined functions.

[0036] Embodiments of the present invention can be executed by a computer processor or a similar device programmed in the form of method steps, or can be executed by an electronic system provided with means for executing these steps. Similarly, an electronic storage device such as a computer disk, CD-ROM, random access memory (RAM), read-only memory (ROM), or a similar computer software storage medium known in the art can be programmed to execute these method steps. Also, electronic signals representing these method steps can be transmitted through a communication network.

[0037] Embodiments of the present invention can be implemented in any conventional computer programming language. For example, the preferred embodiments can be implemented in a procedural programming language (e.g., "C" or "Go") or an object-oriented language (e.g., "C++", "java", "PHP", "PYTHON", or "C#"). Alternative embodiments of the present invention can be implemented as pre-programmed hardware elements, other related components, or a combination of hardware and software components.

[0038] An embodiment can be implemented as a computer program product for use with a computer system. Such an implementation may include a series of computer instructions that are fixed on a tangible medium such as a computer-readable medium (e.g., a floppy disk, CD-ROM, ROM, or fixed disk) or can be transmitted to the computer system via a modem or other interface device such as a communication adapter connected to a network through a medium. The medium can be a tangible medium (e.g., an optical communication line or an electrical communication line) or a medium implemented using wireless technology (e.g., microwave, infrared, or other transmission technologies). The series of computer instructions embody all or part of the functions described hereinbefore. Those skilled in the art should understand that such computer instructions can be written in many programming languages for use with many computer architectures or operating systems. In addition, such instructions can be stored in any storage device, such as a semiconductor, magnetic, optical, or other storage device, and can be transmitted using any communication technology such as optical, infrared, microwave, or other transmission technologies. It is contemplated that such a computer program product can be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink-wrapped software), pre-loaded on a computer system (e.g., on the system ROM or fixed disk), or distributed from a server on a network (e.g., the Internet or the World Wide Web). Of course, some embodiments of the present invention can be implemented as a combination of software (e.g., a computer program product) and hardware. Other embodiments of the present invention can be implemented as entirely hardware, or as entirely software (e.g., a computer program product).

[0039] Those who understand the present invention can now envision alternative structures and embodiments or variations of the above, all of which are intended to fall within the scope of the present invention as defined by the appended claims.

Claims

1. A system for pre-configuring and starting one or more instances of a software module that implements a neural network, the system comprising: - A configuration file that details the hardware and software configurations for implementing multiple instances of the software module; - A process agent module that configures at least one computer system to implement the multiple instances of the software module, and the process agent uses the configuration file to configure the at least one computer system; wherein - The configuration file includes a range of hyperparameter values used by the neural network; - The configuration file is stored in a data storage device such that the configuration file is retrievable and each implementation of the software module uses the configuration file; - The configuration file is uniquely identified by an identifier that is uniquely associated with the software module; The process agent evaluates the outputs of the multiple instances of the software module and determines an optimal set of hyperparameter values for the neural network based on the outputs of the multiple instances of the software module; and Each instance of the multiple instances of the software module uses a different set of hyperparameter values based on the range of hyperparameter values in the configuration file.

2. The system according to claim 1, wherein, The configuration file causes the process agent to start the multiple instances of the software module.

3. The system according to claim 1, wherein, Each instance of the software module produces an output that is sent to the process agent.

4. A method for starting multiple instances of a software module that implements a neural network, the method comprising: Provide a configuration file that details the configuration for implementing the multiple instances of the software module, wherein the configuration file includes a range of hyperparameter values used by the neural network; Provide an identifier that uniquely identifies the configuration file, and the identifier is uniquely associated with the software module; Configure at least one execution environment at least in part based on the configuration file through a process agent module to implement the multiple instances of the software module; Start the multiple instances of the software module; and Determine hyperparameter values for the neural network based on the outputs of the multiple instances of the software module through the process agent module, wherein each instance of the multiple instances of the software module uses a different set of hyperparameter values based on the range of hyperparameter values in the configuration file.

Images