Automatic generation of installation scripts to create source code runtime environment

Abstract

A method for automatically generating an installation script that creates a runtime environment for source code includes: receiving a source code identifier identifying the source code; determining characteristics of a target execution runtime environment in which the source code is to be executed; extracting a dependency identifier from the source code; and providing the dependency identifier and the characteristics of the target execution runtime environment to a trained machine learning (ML) model. The method also includes receiving library version identifiers from the trained ML model, each library version identifier identifying a library that supports a corresponding one of the dependency identifiers and is compatible with characteristics of the target execution runtime environment. An installation script that can be used to create a target execution runtime environment is composed of ML model outputs.

Description

Background Art

[0001] Executing source code in a given computing environment usually requires compiling the source code and compiling all its dependencies in the order in which it is called by the source code. When some dependencies (e.g., libraries) are not available in the computing environment or when the dependencies are compiled in the wrong order, compilation errors are caused. For example, if function A is defined in a first library and includes function calls to functions B and C defined in a second library, then if the second library is not compiled before the first library (in addition, functions B and C are not defined when compiling function A), compilation errors may result. To make things more complicated, the same library can have many different versions, some of which are compatible with some but not all operating systems and certain coding languages, but incompatible with others. For example, a first version of a library may be compatible with Java versions lower than 3.0, and another version of the same library is needed to support Java scripts with versions higher than 3.0.

[0002] For the reasons stated above, creating an execution runtime environment in which source code can run without errors can be time consuming. Summary of the invention

[0003] According to one implementation, the disclosed method provides for receiving source code; determining characteristics of a target execution runtime environment in which the source code is to be executed; extracting dependency identifiers from the source code; and providing the extracted dependency identifiers and the determined characteristics of the computing environment to a trained machine learning (ML) model of an installation script generation engine, the installation script generation engine being configured to utilize semantic associations learned by the ML model to automatically generate an installation script for the source code, which defines dependencies of the source code and can be used to create a target execution runtime environment for the source code.

[0004] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0005] Other implementations are also described and documented herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 An example system is shown that includes an installation script generation engine that automatically generates an installation script that can be used to create an execution runtime environment for a source code product.

[0007] Figure 2 An example system for automatically generating an installation script for a source code product using a trained neural network is shown.

[0008] Figure 3 Various aspects of a training process for creating a multi-layer neural network that can be used in an installation script generation engine that automatically generates installation scripts for source code products are shown.

[0009] Figure 4 Example operations for refining a trained multi-layer neural network to improve the accuracy of associations between source code dependencies and corresponding library version identifiers are shown.

[0010] Figure 5 An example installation script generation engine that automatically generates installation scripts for a source code product is shown.

[0011] Figure 6 Example operations for automatically generating an installation script to create an execution runtime environment for a source code product are shown.

[0012] Figure 7 An example schematic diagram of a processing device suitable for implementing various aspects of the disclosed technology is shown. DETAILED DESCRIPTION

[0013] To execute the source code, the developer performs various operations to create an execution runtime environment, which is a computing environment in which all dependencies of the source code are installed so that the source code can subsequently be compiled without compilation errors. For example, the execution runtime environment can be a local directory in which the source code can be executed, where the directory stores a copy of the source code and all dependencies of the source code. Alternatively, the execution runtime environment can be a virtual machine (VM) configured on a cloud-based server.

[0014] A common way to create a runtime environment for source code is to generate an installation script (e.g., a configuration file) that stores information about the libraries, modules, and data packets used when developing source code. The installation script can be provided to the application that is installed in the dependencies defined in the installation script to generate an execution runtime environment. For example, the Unix operating system accepts an installation file named "requirements.txt" as input to an installation command that is executable to create a runtime environment for a specific source code. Similarly, other platforms can utilize various (multiple) application execution developers to provide installation scripts for creating a runtime environment. In various implementations, the installation script can be a text file (as in the requirements.txt example above) or an executable file that can be executed to create an execution runtime environment for the associated source code.

[0015] Creating an installation script can be a time-consuming and error-prone process. For example, the process requires identifying certain characteristics of the computing environment in which the source code is to be executed (e.g., the operating system and hardware of the machine providing the runtime environment), identifying the libraries called by the source code and all of their dependencies, identifying the correct version of each dependency (e.g., library) that is compatible with the identified characteristics of the target runtime environment, and also correctly defining the dependencies in the installation file with reference to the correct version number and in the correct order to ensure that compilation errors do not occur. Traditionally, developers must manually determine all of the above information and resolve dependency compatibility issues and sequencing issues, such as through trial and error, consulting online forums, etc.

[0016] According to one implementation, the disclosed technology provides an installation script generation engine that uses a trained machine learning model, such as a deep learning network, to automatically generate an installation script for source code when provided with source code and specified characteristics of a target runtime environment. The automatically generated installation script can then be used to create the target runtime environment - automatically created by a service provider who configures the runtime environment, or automatically created by a developer who executes the script. For example, the target runtime environment is a container on a local machine, or alternatively a virtual machine on a local host or on the cloud, where the source code can be compiled without errors.

[0017] Figure 1 An example system 100 is shown that includes an installation script generation engine 102 that automatically generates an installation script for source code. The installation script generation engine 102 is configured to receive certain inputs from a user machine and, in response, automatically generate and test an installation script that can be used to generate an execution runtime environment for source code files using a trained neural network. In one implementation, the installation script generation engine 102 is a cloud-based service hosted by a third-party service provider. In another implementation, some or all aspects of the installation script generation engine 102 are stored on the user machine and / or executed locally on the user machine, such as locally on one of the user devices 104 and 106.

[0018] In one implementation, a code developer interacts with the installation script generation engine 102 through a user interface (eg, a web-based interface) to provide the installation script generation engine 102 with input associated with the source code product. Figure 1 Two different examples of this type of interaction with respect to user devices 104 and 106 are shown.

[0019] exist Figure 1In a simplified example of , it is assumed that the user device 104 is a device to be used to run the source code. In response to receiving a set of inputs from the user device 104, the installation script generation engine 102 creates a corresponding installation script 108, transmits the installation script back to the user device 104, and the developer executes the script on the received installation script to locally create a desired execution runtime environment for the source code. In another implementation, the installation script generation engine 102 creates the installation script 108, configures the virtual machine with certain developer-specified execution runtime characteristics, and creates an execution runtime environment on the virtual machine using the installation script. In this implementation, the developer can be granted access to the execution runtime environment on the virtual machine where the source code can be executed.

[0020] To initiate the generation of the installation script for the source code product, the developer provides certain inputs 112, 114 to the installation script generation engine 102 through a user interface (e.g., a web interface) accessed from the user device 104 or 106. The inputs 112, 114 include at least the source code, or if the source code is publicly available, an identifier for the source code, and certain runtime environment characteristics of the target computing environment in which the source code is to be executed. The specified runtime environment characteristics may include, for example, existing libraries installed on the device and their versions, the version of the operating system executed on the device, the hardware characteristics of the device, such as memory and processor resources, including available cache size (L1 / L2 cache size), processor type (e.g., 32-bit vs. 64-bit, multi-core vs. single-core, GPU vs. CPU, RISC processor vs. CISC processor), the availability of a display device, the size of the L1 and L2 caches on the device, the availability of a display device, etc. In some implementations, the runtime environment characteristics included in the inputs 112, 114 also include an installation script that the developer has created for the source code, such as a script that was not successfully executed. For example, such a script may include libraries called by the source code and corresponding version numbers.

[0021] In the example shown, user device 104 executes 11 operating system. It is worth noting that The operating system 104 includes several supported libraries that may or may not have been installed on the user device 104. In some cases, the user device 104 may store different versions of various libraries and / or be configured to retrieve other (e.g., newer) versions of currently installed libraries or other libraries that have not been installed, such as from a web-based repository.

[0022] The input 112 provided by the user device 104 to the installation script generation engine 102 includes a source code file (which may be written in a variety of different languages), and a set of runtime environment characteristics of the target computing environment in which the source code is to be executed. For example, the set of runtime environment characteristics may include the operating system and various hardware parameters of the user device 104. Upon receiving the input 112, the installation script generation engine 102 determines the coding language of the source code (e.g., Python) and parses the source code to extract certain dependency information (e.g., an "Import [library]" statement or equivalent statement given the selected coding language). Using this information, the installation script generation engine 102 identifies a complete set of corresponding dependencies for the source code, which are determined by the libraries supporting the applicable coding language. And with the operating system ( 11) and is also identified by a version number that is compatible with various specified hardware features of the user device 104.

[0023] The installation script generation engine 102 generates an installation script 108, a test installation script 108 (e.g., Figure 2 ), and—in the illustrated implementation—returns the installation script to the user device 104.

[0024] In the same or another implementation shown with respect to user device 106, installation script generation engine 102 generates another installation script 110, configures a container on virtual machine 118 using the received set of runtime environment characteristics (e.g., included in input 114), and executes installation script 110 in the container to create an execution runtime environment for the source code product on the configured virtual machine 118. For example, input 114 from user device 106 includes the source code product and indicates that the target runtime environment executes Version 10.5. The installation script generation engine 102 parses the source code to determine the applicable coding language (e.g., C++) and extracts dependency information from the source code. The dependency information and other received or determined target runtime environment characteristics (e.g., operating system, hardware specifications, and determined coding language) are used by the installation script generation engine 102 to (1) identify a complete set of corresponding dependency identifiers, including appropriate version numbers that are compatible with the target runtime characteristics, and also (2) identify an appropriate compilation order for the dependencies so that the dependencies can be compiled without errors. Based on the identified dependencies and dependency order, the installation script generation engine 102 generates an installation script 110 and uses the installation script 110 to configure the cloud-based runtime environment on the virtual machine 118.

[0025] In one implementation, the installation script generation engine 102 attempts to generate a runtime environment via a trial-and-error method that provides for configuring multiple different possible runtime environments, each of which is used to attempt to compile the source in order to generate a cloud-based runtime environment for the user as quickly as possible. For example, multiple different runtime environments can be generated in containers on the virtual machine 118, or multiple different runtime environments can be generated in parallel on different containers and / or different virtual machines. When the runtime environment is successfully created, the user who initiated the request can be given access to the execution runtime environment (e.g., on the virtual machine 118) to execute the source code product.

[0026] In one implementation, the installation script generation engine 102 includes a machine learning model, which may be a neural network, trained on a dataset including source code products in various languages, their associated installation scripts, and characteristics of runtime environments in which each of the installation scripts was successfully used to compile the corresponding source code. Figure 2 An example of such a system is shown and described.

[0027] Figure 2 An example system 200 for automatically generating installation scripts for source code products using a trained neural network is shown. System 200 includes an installation script generation engine 202 that is part of a cloud-based service 220 used by developers to create execution runtime environments for their various source code products.

[0028] The installation script generation engine 202 includes an installation script generator 208 and an installation script tester 210. The installation script generator 208 includes a trained machine learning model 218 that performs logic based on semantic associations learned from a corpus of text information included in a training data set 224.

[0029] In different implementations utilizing different types of machine learning models, the training data set 224 may include a variety of different types of information. Figure 2 , training data set 224 includes source code scripts 212, installation scripts 214, and characteristics of the runtime environment 216. For example, a single training input includes three types of information: (1) a source code product; (2) an installation script that is successfully executed to create a runtime environment for the source code product without compilation errors; and (3) various characteristics of the successfully created execution runtime environment (e.g., OS and device hardware characteristics of the device hosting the environment). In other implementations, training data set 224 may also include other information, such as developer comments (e.g., about what worked / didn't work in various different attempts to create the execution runtime environment and compilation errors that may have been observed in those previous attempts).

[0030] The complete set of training inputs in training dataset 224 may include a corpus of source code scripts covering a wide variety of coding languages, corresponding (successful) installation scripts, and hardware and software characteristics of runtime environments in which the installation scripts are successfully run.

[0031] Throughout the training process of the machine learning model 218, the installation script generator 208 learns semantic associations between dependencies (e.g., function calls) referenced in different source code files, library identifiers (e.g., library version numbers), and various runtime characteristics of the environment in which the source code product is successfully compiled given the corresponding library identifiers.

[0032] When provided with a set of source code files 204 and features for a target runtime environment 206, the trained machine learning model 218 uses the learned semantic associations to identify a corresponding set of library identifiers (version numbers) that are ordered in a manner that facilitates error-free dependency compilation and creation of the corresponding execution runtime environment. These library identifiers are then recorded in the installation script according to the identified ordering scheme, as well as in the installation script tester 210 that tests the installation script.

[0033] In one implementation, the installation script tester 210 tests the installation script by first configuring a container on a virtual machine (VM) (e.g., one of the plurality of virtual machines 230), wherein the characteristics 206 of the target runtime environment are determined based on input from the user machine 222. The installation script tester 210 then attempts to run the installation script in the virtual machine container. If the attempt is successful, the execution runtime environment has been created and source code can be compiled without errors in the runtime environment.

[0034] In different implementations, the trained machine learning model 218 can be trained using supervised, unsupervised, or a combination of supervised and unsupervised learning techniques. In one implementation, the trained machine learning model 218 includes a reinforcement learning model. For example, the reinforcement learning model can be initially trained using supervised learning and, over time, refined in an unsupervised manner based on feedback from the installation script tester 210 (e.g., feedback on whether the installation script can be successfully executed to create an execution runtime environment for the source code). In one such implementation, the reinforcement learning algorithm can use previous compilation output and / or comments from developers (e.g., as reinforcement states), successful compilations (e.g., as reinforcement learning rewards), and generate installation sequences as reinforcement learning actions that are performed based on states. The model can be trained using a policy-based or value-based approach.

[0035] If the testing of the installation script results in one or more compilation errors, the installation script tester 210 provides the error(s) back to the installation script generator 208 via the feedback loop 232, and the trained machine learning model 218 modifies the known semantic associations based on the feedback, thereby outputting a modified installation script, which can then be similarly tested by the installation script tester 210. In yet other implementations, the installation script generator 208 outputs multiple different versions of the installation script, and the installation script tester 210 tests the different versions on different virtual machines, each of which is configured with the characteristics 206 of the target runtime environment.

[0036] In one implementation, the trained machine learning model 218 includes one or more deep learning models. For example, the trained machine learning model 218 may include a transformer model, such as a Figure 3 In yet another implementation, the trained machine learning model 218 includes a neural network that was initially generated by a transformer model and then referenced via transfer learning, such as described below with respect to Figure 4 Way of discussion.

[0037] Figure 3 1 shows various aspects of a training process 300 used to create a multi-layer neural network 304 that can be included in an installation script generation engine 302 that automatically generates an installation script for a source code product. The multi-layer neural network 304 can be understood as being included in a plurality of neural networks having the same structure as described above. Figure 1 to Figure 2 The installation script generation engine discussed above may have the same or similar features as the installation script generation engine 302 .

[0038] The multi-layer neural network 304 is created by training a semantic association learning model 306 using a set of input data, which includes information such as that displayed about the sequence input 308. In one implementation, the semantic association learning model 306 is a model trained to transform a given sequence of elements into another sequence of elements, such as by translating a sequence of words in an English sentence into a French sentence. For example, the semantic association learning model 306 is a sequence-to-sequence (Seq2Seq) model or a transformer model, such as BERT (Bidirectional Encoder Representations from Transformers). These models can be pre-trained in an unsupervised manner on very large data sets for language modeling.

[0039] In one implementation, the semantic association learning model 306 accepts as input a sequence input 308. The sequence input 308 represents a single training input based on at least three types of information—(1) source code script 212; (2) installation script 214 (e.g., a script that successfully creates a runtime environment for the source code script); and (3) characteristics of the runtime environment in which the installation script 214 is successfully executed, such as the operating system and hardware characteristics of the host device. In one implementation, the sequence input 308 is a large block of text that includes the source code script 312, followed by the installation script 314, followed by the corresponding characteristics 316 of the runtime environment. Different segments (e.g., information types) in the sequence input 308 can be separated by tokens (such as SEP and CLS tokens), which are used to help the installation script generation engine 302 understand the end of one input and the beginning of another input in the same sequence input 308.

[0040] The installation script generation engine 302 receives the sequence input 308 and can generate certain embeddings from the received information. For example, the BERT model accepts three embeddings as input, which can be generated from the sequence input 308. The three embeddings each have a dimension corresponding to the number of words in the sequence input 308, and include (1) a word-element embedding, where each index represents a corresponding word from the sequence input 308 that matches the index; (2) a segment embedding, where each index includes a segment number (e.g., a sentence number in an input sentence block) within which the corresponding word at the index is included; and (3) a position embedding, where each index represents the position of the corresponding word at the index within its associated segment of the sequence input 308.

[0041] In contrast to directional models that read text input sequentially, Transformer models such as BERT read the entire sequence of words at once and are called “bidirectional” for this reason. This property allows the model to learn the context of a word based on all of the word’s surroundings.

[0042] Training the semantic association learning model 306 on a large corpus of source code files and their corresponding installation scripts and runtime features produces a multi-layer neural network 304, which serves as the logical core of the installation script generation engine 302. The multi-layer neural network 304 is essentially a logical network with many layers and nodes that embeds an understanding of how each distinct semantic element in the corpus of training input information is related to every other element.

[0043] The training data used to train the semantic association learning model 306 can be obtained by using crowdsourcing techniques and / or by cleaning online sources (e.g. ) or other suitable methods. In order to better summarize the trained multi-layer neural network 404, the training data may include source code from multiple programming languages ​​and for different operating systems.

[0044] In one implementation, the transfer learning model is used to refine the top layers of the multi-layer neural network 304 to provide more accurate next segment predictions for individual customers. Figure 4 Examples of such systems are discussed in further detail.

[0045] Figure 4 A method for refining a trained multi-layer neural network 404 (such as Figure 3 4. The example operation 400 of the multi-layer neural network 304 of the embodiment of the present invention is to improve the accuracy of the association between the source code dependency and the supporting library version identifier. The trained multi-layer neural network 404 is shown in the installation script generation engine 402, which can be understood as having the same or similar characteristics as the installation script generation engine described in other figures herein. The installation script generation engine 402 includes multiple models, which are used in combination to create the trained multi-layer neural network 404.

[0046] Although the trained multi-layer neural network 404 may be generated using a variety of different types of known supervised learning and unsupervised learning techniques in different implementations, the operations 400 provide for first training a generalized semantic association learning model 406 and then refining the top layers of the trained multi-layer neural network 404 using a transfer learning model 408 .

[0047] The initial training of the semantic association learning model 406 can be similar to the above Figure 3 The transfer learning model 408 is performed in the same or similar manner as described for the semantic association learning model 306 of . This produces a rich learned semantic association network that covers many programming languages ​​and computing environments. Thereafter, the transfer learning model 408 refines the trained multi-layer neural network 404 using an additional set of "refined" training inputs. In one implementation, these refined training inputs are specific to a single customer. For example, the refined training inputs may include a large number of source code files written by a customer and corresponding successfully executed installation scripts and corresponding execution runtime characteristics. Since customers may often code in the same (multiple) languages ​​and / or often reuse certain libraries and certain hardware configurations, this customer-specific retraining can significantly improve the quality of predictive associations generated by the trained multi-layer neural network 404 regarding inputs associated with the same customer.

[0048] The transfer learning model employs supervised learning to transfer knowledge from one supervised learning task to another supervised learning task. In the illustrated implementation, transfer learning is used to teach a trained multi-layer neural network a set of "correct" associations for an individual customer. For example, a "correct" association can be an association between an individual "Import [library]" in a customer-provided source code file and a library version identifier compatible with the customer's target runtime features, so that the import statement can be executed to compile the corresponding library without errors in the customer-specified runtime environment. This concept is illustrated by customer-specific training data 410.

[0049] Customer-specific training data 410 includes multiple training input sets provided by a single individual customer (e.g., a corporate entity that produces many software products). Each individual customer-provided input within the customer-provided input set includes a customer source code file 412, an installation script 414 that is used to successfully create a runtime environment for the customer source code file 412, and a set of runtime characteristics 416 related to the machine and / or software configuration (e.g., operating system, processor type) that supports the runtime environment.

[0050] The input to the transfer learning model 408 is more fine-grained than the customer-provided input described above and is also related to Figure 3 Related to the semantic association learning model 306. Instead of accepting the entire source code file and the corresponding installation script as training input, the transfer learning model 408 accepts dependency-specific sentence blocks that include information related to a single source code product dependency.

[0051] According to one implementation, preprocessing of each training input set in customer-specific training data 410 includes extracting dependency identifiers from customer source code files 412, and for each extracted dependency identifier, extracting a corresponding library version identifier from an associated installation script 414. For example, the Python coding language uses an "import [library]" statement in a source code product to identify and import each library required to support one or more functions referenced in a corresponding source code body. Thus, the "Import [library]" statement is an example of a dependency identifier 424. Other examples include the name of a subroutine called from customer source code file 412.

[0052] Thus, dependency-specific training inputs 418, 420, 422 can be generated, each dependency-specific training input comprising an extracted dependency identifier (e.g., dependency identifier 424), a corresponding correct library version identifier (e.g., library version identifier 426), and one of the runtime environment characteristics 428 associated with a previously successfully created execution runtime environment for the customer source code file 412.

[0053] In one implementation, the transfer learning model 408 receives a set of training inputs for the source code file 412, where each training input in the set is in the form of [sentence 1], [sentence 2]. For example, sentence 1 includes the extracted import statement 242 and the characteristics of the runtime environment 216 (e.g., as a long text string), and sentence 2 represents the corresponding correct library version identifier extracted from the installation script 414. For example, the format of the input is similar to: [import Pandas, runtime characteristics], [Pandas 1.2], where "Pandas" is the library called by the source code file, "Pandas 1.2" is the correct version identifier for the library, and "runtime characteristics" is a text string indicating the characteristics of the execution runtime environment (e.g., operating system and computer hardware), where the Pandas library is successfully compiled using library version 1.2.

[0054] These training inputs can be transformed into a set of input embeddings as required by the particular transfer learning model employed. In one embodiment, One Hot Encoded embeddings are used to represent import statements when training the transfer learning model. Using the training techniques described above, a dictionary can be generated to map each import data package to a library with the correct version identifier for the customer's runtime environment characteristics.

[0055] It is worth noting that Figure 3 and Figure 4 The described neural network training method (e.g., use of unsupervised training utilizing a generalized transformer model, followed by customer-specific refinement by supervised training utilizing a transfer learning model) represents one method of many different suitable methods for creating the underlying logic network utilized by the installation script generation engine 402 to automatically generate source code installation scripts. Other suitable methods will be apparent to those skilled in the art.

[0056] about Figure 5 Example functionality of the installation script generation engine 402 is discussed.

[0057] Specifically, Figure 5 An example installation script generation engine 502 is shown that automatically generates installation scripts for a source code product. In some implementations, the installation script generation engine 502 also uses each automatically generated installation script to configure an execution runtime environment on a virtual machine for use by a developer when executing the source code product.

[0058] The installation script generation engine 502 includes an input block generator 513, an installation script generator 511, and an installation script tester 512. The input block generator 513 receives input from a developer, which is related to the creation of an execution runtime environment for a specific source code product. In one implementation, the developer provides the source code product 506 and certain specified characteristics 508 of the target runtime environment (e.g., the operating system and hardware characteristics of the machine to execute the source code product 506) to the installation script generation engine 502 through a web-based interface. In some implementations, one or more characteristics 508 of the target runtime environment are passively retrieved (e.g., without developer input) from the developer's computing device by the input block generator 513.

[0059] The input block generator 513 parses the source code product 506 and extracts dependency identifiers (e.g., "Import [Library]" statements). For each different extracted dependency identifier, the input block generator 513 creates a sentence input block 510, which is provided as input to the transfer learning model 518. An example sentence input block of the sentence input block 510 is a text string that includes a single import statement from among the extracted import statements, followed by one or more features from among the features 508 of the target runtime environment.

[0060] The installation script generator 511 may include a transfer learning model 518 that is capable of converting each of the received sentence input blocks 510 for the source code product 506 into an associated library version identifier that is compatible with the characteristics 508 of the target runtime environment. For example, the transfer learning model 518 may receive three sentence input blocks 510, each sentence input block being associated with a different import statement (e.g., Import pandas; Import numpy; Import tensorflow-hub) extracted from the source code product 506. For each input block statement, the transfer learning model maps the import statement to a corresponding library version identifier (e.g., "Pandas 1.2, Numpty 1.17.3, Tensorflow-hub>=0.7.0). This mapping by the transfer learning model 518 is performed using the logical semantic associations of the trained multi-layer neural network 504.

[0061] The trained multi-layer neural network 504 is created through one or more machine learning training processes, such as Figure 3 and Figure 4Although different types of models can be used to initially create the trained multi-layer neural network 504, the illustrated implementation uses a transformer model 514 to learn a general corpus of information related to source code products, libraries, languages, and runtime characteristics. The top few layers of the network are then refined, such as by training the transformer model 514 to modify the trained multi-layer neural network based on associations learned from a customer-specific training data set, such as with respect to Figure 4 Way of discussion.

[0062] After the transfer learning model 518 identifies the library version identifier associated with each "Import" statement extracted from the source code, logic can be employed to identify the correct ordering of the library version identifiers in the resulting installation script 522 so that the installation script 522 can be compiled without errors. In one implementation, the installation script generator 511 includes a first trained ML model that identifies the dependencies required to compile the source code (as described above), and a second ML model (not shown) that determines the ordering of the dependencies. For example, the second model can be trained on inputs including various source code scripts and characteristics of the execution runtime environment in which the source code scripts are successfully compiled. Given the set of determined dependencies for the source code product and the characteristics 508 of the target runtime environment as input, the second model can reference the body of training data to determine the correct order of the dependencies.

[0063] Using the output of the transfer learning model 518 and the determined order of dependency identifiers, the script composer 520 composes an installation script 522 for the source code product. The generated installation script 522 is tested by the installation script tester 512. In one implementation, the script composer 520 generates multiple different versions of the installation script 522, which are tested in parallel on different virtual machines by the installation script tester 512. Reinforcement learning can be used to refine the associations in the trained multi-layer neural network 504 based on feedback from the installation script tester 512 about the successful or unsuccessful creation of the configuration of the target execution runtime environment for the source code product 506.

[0064] In some implementations, the installation script generator 511 includes a rule-based model that runs on top of or in parallel with a trained machine learning model (e.g., transfer learning model 518). For example, the rule-based model can apply static rules based on various developer inputs about configurations that succeeded or failed for a specified runtime environment. Testing of existing rule-based models indicates that such models may succeed in a large number of cases (e.g., a 60% success rate). For this reason, there may be some instances where the installation script 522 can be generated more quickly using a rule-based model, falling back to the trained machine learning model in instances where the rule-based model fails.

[0065] When installation script tester 512 determines that installation script 522 is valid (e.g., the script can be executed on a machine configured with features 508 of the target runtime environment to create a source code runtime environment without compilation errors), the operation of installation script generation engine 502 is completed. Installation script 522 can be returned to the developer for local execution, or alternatively, the developer can be provided with access to the source code runtime environment that has been created by installation script tester 512 for source code product 506. For example, the developer can be provided with access to a container on a virtual machine configured with features 508 of the target runtime environment in which the installation script has been successfully executed to install the dependencies of source code product 506.

[0066] Figure 6 Example operations 600 are shown for automatically generating an installation script to create an execution runtime environment for a source code product. Receiving operation 602 receives source code, such as source code from a user device or a public repository, based on a source code identifier provided by a developer (eg, via a web interface).

[0067] A determination operation 604 determines characteristics of a target runtime environment in which the source code is to be executed. The determination operation 604 may include receiving one or more of the characteristics (e.g., input provided by a developer) through a web portal. For example, a developer may specify one or more characteristics of a target execution runtime environment in which the source code is to be run. In another implementation, the determination operation 604 includes retrieving one or more of the characteristics from a user device (e.g., querying the device to identify the device's operating system, processor type, and / or other hardware characteristics). A parsing and extraction operation 606 parses the source code and extracts a dependency identifier from the source code. For example, the dependency identifier is an "import [library]" statement, i.e., the name of a subroutine called by the source code.

[0068] A providing operation 608 provides the extracted dependency identifiers and the determined characteristics of the target execution runtime environment to the trained machine learning model. In one implementation, the machine learning model is included in an installation script generation engine that is configured to automatically generate an installation script for the source code using the semantic associations learned by the ML model.

[0069] A receiving operation 610 receives an output from the trained ML model including a library version identifier corresponding to each of the extracted dependency identifiers, the library version identifier being compatible with the determined characteristics of the target execution runtime environment. For example, the dependency identifier may be a library named "numpy", and the corresponding library version identifier is numpy3.0, which is compatible with the operating system and processor type included in the determined characteristics of the target execution runtime environment.

[0070] The composition operation 612 composes an installation script for the source code that includes the output generated by the trained machine learning model. In one implementation, the machine learning model performs the composition operation 612. In other implementations, the supplementation module performs the composition operation 612 based on several rounds of inputs provided to the machine learning model and outputs received from the machine learning model.

[0071] An execution operation 614 executes the automatically generated installation script to create a target execution runtime environment for the source code. In one implementation, the execution operation 614 is performed during testing of the installation script. For example, the installation script can be tested in a container on a virtual machine configured with the determined characteristics of the target execution runtime environment. After successfully executing the installation script, access to the container can be provided to the developer. In another implementation, the automatically generated installation script is provided to the developer, and the developer performs operations to locally install the dependencies listed in the installation script in a scripted order.

[0072] Figure 7 An example schematic diagram of a processing device 700 suitable for implementing various aspects of the disclosed technology is shown. In one implementation, the processing device 700 is a computer that executes an installation script generation engine (e.g., Figure 1 The server uses an installation script generation engine 102 to automatically generate an installation script (e.g., a requirements.txt file) that can be used to create an execution runtime environment for a given source code product.

[0073] The processing device 700 includes a processing system 702, memory device(s) 704, a display 706, and other interfaces 708 (e.g., buttons). The memory device(s) 704 typically include volatile memory (e.g., random access memory (RAM)) and non-volatile memory (e.g., flash memory). An operating system 710 may reside in the memory device(s) 704 and be executed by the processing system 702. One or more applications 712 (such as the installation script generation engine 102 or its submodules (e.g., as described with respect to FIG. 1 ) may be executed by the processing system 702. Figure 5 The described)) may be loaded into memory and executed by the processing system 702 on the operating system 710.

[0074] Processing device 700 includes power supply 716, which is powered by one or more batteries or other power sources, and which provides power to other components of processing device 700. Power supply 716 may also be connected to an external power source that overlays or recharges the internal batteries or other power sources.

[0075] In addition, the processing device 700 includes one or more communication transceivers 730 and antennas 732 to provide network connectivity (e.g., mobile phone networks, ). The processing device 700 may also be coupled to various input devices 734, such as a microphone, a keyboard, a touch display, etc. In implementation, the installation script generation engine and various other applications and other modules and services are embodied by instructions stored in (multiple) memory devices 704 and / or storage devices 728 and processed by the processing system 702. (Multiple) memory devices 704 may be the memory of a host device or the memory of an accessory coupled to the host. The installation script generation engine may include a trained multi-layer neural network, which is stored in (multiple) memory devices 704 or in the memory of one or more other computing devices (e.g., various interconnected processing nodes), which are communicatively coupled to the processing device 700, such as via the Internet.

[0076] The processing device 700 may include various tangible computer-readable storage media and intangible computer-readable communication signals. Tangible computer-readable storage may be embodied by any available media, and includes both volatile and non-volatile storage media, removable and non-removable storage media, which can be accessed by the processing device 700. Tangible computer-readable storage media do not include intangible and transient communication signals, but include volatile and non-volatile, removable and non-removable storage media implemented in any method or technology for storing such as computer-readable instructions, data structures, program modules or other data. Tangible computer-readable storage media include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CDROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other tangible media that can be used to store the desired information and can be accessed by the processing device 700. In contrast to tangible computer-readable storage media, intangible computer-readable communication signals may embody computer-readable instructions, data structures, program modules, or other data residing in a modulated data signal such as a carrier wave or other signal transmission mechanism. The term "modulated data signal" means a signal having one or more of its characteristic sets that are set or changed in such a way as to encode information in the signal. By way of example, intangible communication signals include wired media such as a wired network or a direct-wired connection and wireless media such as acoustic, RF, infrared and other wireless media.

[0077] Some embodiments may include an article of manufacture. An article of manufacture may include a tangible storage medium storing logic. Examples of storage media include one or more types of computer-readable storage media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. Examples of logic include various software elements, such as software components, programs, applications, computer programs, applications, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, processes, software interfaces, application program interfaces (APIs), instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols, or any combination thereof. For example, in one implementation, an article of manufacture stores executable computer program instructions that, when executed by a computer, cause the computer to perform methods and / or operations according to the described embodiments. Executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Executable computer program instructions can be implemented according to a predefined computer language, method or syntax to instruct a computer to perform specific functions. Instructions can be implemented using any suitable high-level, low-level, object-oriented, visual, compiled and / or interpreted programming language.

[0078] The following overview provides a non-exhaustive set of illustrative examples of the techniques described herein. According to a first aspect, some implementations provide a method for automatically generating an installation script for creating a runtime environment for source code. The method includes:

[0079] Receiving a source code identifier identifying source code; determining characteristics of a target execution runtime environment for the source code; extracting dependency identifiers from the source code; providing the dependency identifiers and the characteristics of the target execution runtime environment to a trained machine learning (ML) model; receiving library version identifiers from the trained ML model, each library version identifier identifying a library that supports a corresponding one of the dependency identifiers and is compatible with the characteristics of the target execution runtime environment; and composing an installation script including the library version identifiers, the installation script being usable to create the target execution runtime environment for the source code.

[0080] The method of A1 above is advantageous because it eliminates the time-consuming trial and error process that developers currently perform when generating source code installation scripts.

[0081] (A2) In some implementations of A1, the installation script is an executable file that generates a target execution runtime environment. (A3) In other implementations of A1, the installation script is a text file that is input to an installation function, which uses the installation script to create the target execution runtime environment.

[0082] (A4) In yet other implementations of A1-A3, the characteristic of the target execution runtime environment is a characteristic of the user's machine, and the method further comprises providing an installation script to the user's machine. The method of A4 is advantageous because it allows the user to receive a ready-made installation script tailored to the user's machine that can be executed locally without error.

[0083] (A5) In other implementations of A1-A4, the method further includes configuring a container on the virtual machine using characteristics of the target execution runtime environment; creating an execution runtime environment for the source code in the container on the virtual machine using an installation script; and in response to successfully creating the execution runtime environment in the container on the virtual machine, providing access to the container to the user device. The method of A5 is advantageous because it provides automatic generation of a runtime environment on the virtual machine, which the user can access to run the source code without having to perform an affirmative operation of executing the installation script.

[0084] (A6) In other implementations of A1-A5, determining the characteristics of the target execution runtime environment includes detecting the characteristics of the target execution environment from the user device. (A7) In other implementations of A1-A6, the method also includes generating multiple different versions of the installation script; configuring multiple virtual machines using the characteristics of the target execution runtime environment; and testing each of the different versions of the installation script on the configured multiple virtual machines in parallel. The method of A7 is advantageous because it can provide accelerated generation of the target runtime environment compared to implementations that continuously create and test the installation script (e.g., recreate and retest when the installation script fails to generate the target runtime environment).

[0085] (A8) In other implementations of A1-A7, the installation script specifies an order of dependencies identified by dependency identifiers that facilitates compiling the dependencies without errors.

[0086] (A9) In yet other implementations of A1-A8, the trained ML model includes at least one of a transformer model and a transfer learning model.

[0087] In another aspect, some implementations include a computing system for automatically generating an installation script that creates a runtime environment for source code. The computing system includes a hardware logic circuit device configured to perform any of the methods described herein (eg, Methods A1-A9).

[0088] In another aspect, some implementations include a computer-readable storage medium for storing computer-readable instructions. The computer-readable instructions, when executed by one or more hardware processors, perform any of the methods described herein (eg, Methods A1-A9).

[0089] The above description, examples and data provide a complete description of the structure and use of example implementations. Since many implementations can be made without departing from the spirit and scope of the claimed invention, the claims appended hereto define the present invention. In addition, the structural features of different examples can be combined into another implementation without departing from the claims recorded.

Claims

1. A method for automatically generating an installation script, wherein the installation script creates a runtime environment for source code, the method comprising: receiving a source code identifier identifying the source code; determining characteristics of a target execution runtime environment for the source code; extracting dependency identifiers from the source code; providing the dependency identifier and the characteristic of the target execution runtime environment to a trained machine learning (ML) model; receiving library version identifiers from the trained ML model, each library version identifier identifying a library that supports a corresponding one of the dependency identifiers and is compatible with the feature of the target execution runtime environment; as well as An installation script including the library version identifier is composed, the installation script being usable to create the target execution runtime environment for the source code. 2 . The method of claim 1 , wherein the installation script is an executable file that generates the target execution runtime environment.

3. The method of claim 1, wherein the installation script is a text file that is input to an installation function, the installation function using the installation script to create the target execution runtime environment.

4. The method of claim 1 , wherein the characteristic of the target execution runtime environment is a characteristic of a user machine, and the method further comprises: The installation script is provided to the user machine.

5. The method according to claim 1, wherein the method further comprises: configuring a container on a virtual machine using the characteristics of the target execution runtime environment; Creating the execution runtime environment for the source code in the container on the virtual machine using the installation script; as well as In response to successfully creating the execution runtime environment in the container on the virtual machine, providing access to the container to a user device. 6 . The method of claim 1 , wherein determining the characteristic of the target execution runtime environment comprises detecting the characteristic of the target execution environment from a user device.

7. The method according to claim 1, further comprising: generating a plurality of different versions of the installation script; configuring a plurality of virtual machines using the characteristics of the target execution runtime environment; Each of the different versions of the installation script is tested in parallel on the configured plurality of virtual machines.

8. The method of claim 1, wherein the installation script specifies an order of dependencies identified by the dependency identifiers, the order facilitating error-free compilation of the dependencies.

9. The method of claim 1, wherein the trained ML model comprises at least one of a transformer model and a transfer learning model.

10. A system for automatically generating an installation script, wherein the installation script creates a runtime environment for source code, the system comprising: Processing systems; Memory; An installation script generation engine is stored in the memory and is executable by the processing system to: receiving a source code identifier identifying the source code from a device; receiving, from the device, characteristics of a target execution runtime environment for the source code; extracting dependency identifiers from the source code; providing the dependency identifier and the characteristic of the target execution runtime environment to a trained machine learning (ML) model; receiving library version identifiers from the trained ML model, each library version identifier identifying a library that supports a corresponding one of the dependency identifiers and is compatible with the feature of the target execution runtime environment; as well as An installation script including the library version identifier is composed, the installation script being usable to create the target execution runtime environment for the source code.

11. The system of claim 10, wherein the characteristic of the target execution runtime environment is a detected characteristic of the device.

12. The system of claim 10, wherein the characteristic of the target execution runtime environment is a characteristic of the device, and the installation script generation engine is further configured to: The installation script is provided as output to the device.

13. The system of claim 10, wherein the characteristic of the target execution runtime environment is a user-specified characteristic, and the installation script generation engine is further executable to: configuring a container on a virtual machine using the characteristics of the target execution runtime environment; Using the installation script to create the execution runtime environment for the source code in the container on the virtual machine; as well as In response to successfully creating the execution runtime environment in the container on the virtual machine, providing the device with access to the container.

14. The system of claim 10, wherein the installation script generation engine is further executable to: generating a plurality of different versions of the installation script; configuring a plurality of virtual machines using the characteristics of the target execution runtime environment; testing each of the different versions of the installation script in parallel on the configured multiple virtual machines; as well as In response to successful testing of the particular version of the installation script on the virtual machine, access is provided to a runtime execution environment created on the virtual machine by the successful testing.

15. The system of claim 10, wherein the installation script specifies an order of dependencies identified by the dependency identifiers, the order facilitating error-free compilation of the dependencies.

Images