Programming language for interacting with large language model

Abstract

To provide a programming language for interacting with a large language model.SOLUTION: In an example, a software package is provided that provides an interface so that when software is executed, a user is able to provide a natural language prompt using a scripting language that directs the interface to interact with a large language model (LLM), adding contextual information to prompts sent to the LLM. The interface is then also capable of extracting and evaluating a programming code generated by the LLM, so that the generated code may be used immediately in a software package without the need for user editing and / or copying / pasting.SELECTED DRAWING: None

Description

[Technical field]

[0001] This document relates generally to computer systems. More particularly, this document relates to the use of large-scale language models (LLMs). [Background technology]

[0002] LLM refers to artificial intelligence (AI) systems that are trained on large datasets to understand and generate human language. These models are designed to process and understand natural language in a way that allows the models to answer questions, converse, generate text, and perform a variety of language-related tasks. Summary of the Invention [Means for solving the problem]

[0003] The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals indicate similar elements and in which: [Brief description of the drawings]

[0004] [Figure 1] FIG. 1 is a block diagram illustrating a system for automatically generating software programming code in accordance with an illustrative embodiment. [Diagram 2] FIG. 1 is a flow diagram illustrating a method for automatically generating a portion of software programming code according to an example embodiment. [Diagram 3] FIG. 2 is a block diagram illustrating a software architecture that may be installed on any one or more of the devices described above. [Figure 4] FIG. 1 is a schematic diagram of a machine in the form of a computer system, in which a set of instructions can be executed to cause the machine to perform any one or more of the methods discussed herein, according to an exemplary embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] In the following description, exemplary systems, methods, techniques, instruction sequences, and computing machine program products are discussed. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of various exemplary embodiments of the present subject matter. However, it will be apparent to one skilled in the art that various exemplary embodiments of the present subject matter may be practiced without these specific details.

[0006] Generating programming code to create new software applications can be laborious and time-consuming. LLMs offer the opportunity to greatly speed up this process by allowing computer models to create programming code based on natural language user prompts. However, technical challenges are encountered when using LLMs to create programming code. Specifically, the generated code typically has errors that the user must manually and repeatedly correct, often because the LLM cannot utilize contextual information that would help make the code better suit the user's intent or conform to the syntax of the desired programming language. Furthermore, using the generated code requires extensive copying and pasting by the user.

[0007] In one exemplary embodiment, a software package is provided that provides an interface such that, when the software is executed, a user can provide natural language prompts using a scripting language that directs the interface to interact with the LLM by adding contextual information to the prompts sent to the LLM. The interface can then also extract and evaluate programming code generated by the LLM, thereby allowing the generated code to be immediately used within the software package without the need for user editing and / or copy / paste.

[0008] LLMs used to generate information are commonly referred to as generative artificial intelligence (GAI) models. GAI models can be implemented as generative pre-trained transformer (GPT) models or bidirectional encoders. GPT models are a type of machine learning model that uses a transformer architecture, a type of deep neural network that excels at processing sequence data such as natural language.

[0009] A bidirectional encoder is a type of neural network architecture in which an input sequence is processed in two directions: forward and backward. In the forward direction, the encoder starts from the beginning of the sequence and processes the input one token at a time, while in the backward direction, the encoder starts from the end of the sequence and processes the input in reverse order.

[0010] By processing the input sequence in both directions, a bidirectional encoder can capture more context information and dependencies between words, resulting in better performance.

[0011] A bidirectional encoder can be implemented as a Bidirectional Long Short-Term Memory (BiLSTM) model or a Bidirectional Encoder Representations from Transformers (BERT) model.

[0012] Each direction has its own hidden state, and the final output is a combination of these two hidden states.

[0013] Long short-term memory (LSTM) is a type of recurrent neural network (RNN) designed to overcome the vanishing gradient problem in traditional RNNs, which can make it difficult to learn long-term dependencies in sequence data.

[0014] LSTMs contain a cell state, which acts as a memory that stores information over time. The cell state is controlled by three gates: an input gate, a forget gate, and an output gate. The input gate determines how much new information is added to the cell state, while the forget gate determines how much old information is discarded. The output gate determines how much of the cell state is used to compute the output. Each gate is controlled by a sigmoid activation function, which outputs a value between 0 and 1 that determines how much information passes through the gate.

[0015] In BiLSTM, there are separate forward and backward LSTMs. At each time step, the forward LSTM cell and the backward LSTM cell receive the current input token and the hidden state from the previous time step. The forward LSTM processes the input token from left to right, while the backward LSTM processes the input token from right to left.

[0016] The output of each LSTM cell at each time step is a combination of the input token and the previous hidden state, allowing the model to capture both short-term and long-term dependencies between the input tokens.

[0017] BERT applies bidirectional training of models known as transformers to language modeling, in contrast to prior art solutions that examine text sequences left-to-right or right-to-left. Bidirectionally trained language models have a deeper understanding of the context and flow of language than unidirectional language models.

[0018] More specifically, the transformer encoder reads an entire sequence of information at once and is therefore considered bidirectional (although it could actually be said to be non-directional). This feature allows the model to learn the context of a piece of information based on all of its surroundings.

[0019] In another exemplary embodiment, a Generative Adversarial Network (GAN) embodiment can be used. A GAN is a supervised machine learning model with two sub-models: a generator model trained to generate new examples, and a classifier model that tries to classify examples as either real or artificial. The two models are trained together in an adversarial fashion (using a zero-sum game from game theory) until the classifier model is fooled roughly half the time, which means that the generator model is generating plausible examples.

[0020] The generator model takes as input a fixed-length random vector and generates samples in the domain of interest. This vector is drawn randomly from a Gaussian distribution and is used to seed the generation process. After training, the points in this multidimensional vector space correspond to points in the problem domain, forming a compressed representation of the data distribution. This vector space is called the latent space, or the vector space of latent variables. Latent variables, or hidden variables, are variables that are important to a domain but cannot be directly observed.

[0021] The classifier model takes as input examples from this domain (real or artifact) and predicts a binary class label: real or fake (artificial).

[0022] While generative modeling is an unsupervised learning problem, a neat property of GAN architectures is that training a generative model is framed as a supervised learning problem.

[0023] Two models (generator and discriminator) are trained together: the generator produces batches of examples, and these, along with real examples from the domain, are fed to the discriminator, which classifies them as real or fake.

[0024] The classifier is then updated to become better at distinguishing real from fake samples in the next round, and importantly, the generator is updated based on how well the generated samples fool the classifier.

[0025] In another exemplary embodiment, the GAI model is a variational autoencoder (VAE) model. A VAE comprises an encoder network that compresses input data into a lower-dimensional representation called a latent code, and a decoder network that generates new data from the latent code.

[0026] Regardless of the form of the GAI model, it includes a generative classifier, which may be implemented, for example, as a Naive Bayes classifier.

[0027] The solution works with any type of GAI model, but is described in one implementation for use with the GPT model in particular. In one such implementation, the programming language provided by the interface may be called PromptScript, because it is designed to interact with the GPT model via prompts and it is a scripting language. In some exemplary embodiments, this scripting language may be built on top of an existing scripting language, such as JavaScript, but that is not required.

[0028] Furthermore, in one exemplary embodiment, a caching mechanism is introduced that caches the generated code so that if the same natural language prompt written in a scripting language (e.g., PromptScript) is subsequently provided, rather than requiring the code to be generated again, the previously generated code can be retrieved from the cache and utilized, thereby greatly speeding up the execution of future invocations of the prompt.

[0029] The solution also works with any type of application developer software.

[0030] Cloud computing can be described as Internet-based computing that provides shared computer processing resources and data to computers and other devices on demand. Users can establish respective sessions during which processing resources and bandwidth are utilized. During a session, for example, a user is provided on-demand access to a shared pool of configurable computing resources (e.g., computer networks, servers, storage space, applications, and services). The computing resources can be provisioned and released (e.g., scaled) to meet the demands of the users.

[0031] A common architecture in cloud platforms includes services (also called microservices), which have become popular in service-oriented architectures (SOA). In such SOAs, applications consist of multiple independent services. These services are deployed and managed within the cloud platform and run on top of the cloud infrastructure. In some examples, service-based applications can be created and / or extended using application programming frameworks. In an exemplary embodiment, created software servers implement the services.

[0032] FIG. 1 is a block diagram illustrating a system 100 for automatically generating software programming code, according to an exemplary embodiment. Here, an application development server 102 provides a server that enables one or more clients, such as an application development client 104, to create and manage software servers, such as an OData server for web services. The application development client 104 is typically used to control the application development server 102 to create applications using a first programming language, and the application development client 104 may be where a user creates a software application 106 (although in such a case the software application 106 will typically also be stored on the application development server 102 as a backup and / or so that other users can edit it). However, this aspect is not limiting, and in some exemplary embodiments the software application 106 is created solely on the application development server 102. The software application 106, which is essentially in the form of a programming language until it is compiled or interpreted, typically includes a library 108 that includes programming language specific files 110. For example, if a user wishes to use the application development client 104 to create an application written in JavaScript, the programming language specific files are JavaScript libraries.

[0033] In an exemplary embodiment, a dedicated programming language is added on top of the programming language in which the user wishes to program, here JavaScript, and thus an interface package, referred to herein as PromptScript package 112, is included within programming language specific files 110 in library 108. PromptScript package 112 contains all the files necessary to create and run the LLM programming language interface described herein.

[0034] Thus, when PromptScript package 112 is executed, it launches PromptScript interface 114 with which a user can communicate to generate application code using LLM 116 .

[0035] As mentioned above, a file system cache 118 may be provided that caches code generated by the LLM 116 for future invocations.

[0036] More specifically, PromptScript interface 114 receives a prompt (optionally including one or more test functions) from a user (via application development client 104). First, file system cache 118 is checked to determine whether the same prompt has been previously received and processed. This may be accomplished, for example, by applying a hash function to the provided prompt and determining whether the hash generated by the hash function is already stored in the system cache. If so, the generated code associated with the hash may be returned from file system cache 118 to PromptScript package 112 for use as part of the application code that the user is building.

[0037] If not, the PromptScript interface 114 appends a predefined system message to the prompt and sends the combined system message / prompt (including any provided test functions) to the LLM 116, which generates code based on the combined system message / prompt. The PromptScript interface 114 then parses this code. The system message may have instructed the LLM 116 to return the code inside a markdown fence. A Markdown fence, also known as a code fence or code block, is a syntactic feature in which a block of code is highlighted by inserting special characters before and after the block of code. For example, a backtick (`) or a tilde (~) can be inserted at the beginning and end of a code block to demarcate the code.

[0038] In that case, PromptScript interface 114 can parse the returned code by looking for a fence, which allows PromptScript interface 114 to discard any extraneous text returned outside the fence. Thus, for example, LLM 116 might return some generated JavaScript code inside a fence, along with some extraneous text before and after the JavaScript code, in which case PromptScript interface 114 can use the fence to extract the JavaScript code. However, if no such fence is found, PromptScript interface 114 can accept the returned code as is.

[0039] PromptScript interface 114 then validates the code using programming language-specific validation functions. JavaScript includes a function called "eval" that evaluates whether the provided code meets JavaScript's syntactic and other programming requirements and then executes the code. The eval function returns the result of whatever expression is in the input string. If the code is invalid, the eval function returns an error. Test functions run the generated piece of code with specified inputs and the specified desired output. Each test function provides a way for PromptScript interface 114 to determine whether the generated code meets the user's intent for the generated code.

[0040] As an example, a user may wish to generate a Fibonacci function written in JavaScript that, when executed, generates a value in the Fibonacci sequence at a position specified by the input to the function. Thus, for example, a user may wish the Fibonacci function to return "3" if the input is "4" because "3" is in the fourth position of the Fibonacci sequence (assuming the initial position in the sequence is assigned position 0). If a user inputs a simple "Generate a Fibonacci function written in JavaScript" prompt into the PromptScript interface 114, the generated function may not satisfy the user's intent for the function. For example, the generated function may return a function that outputs the entire sequence of values ​​in the Fibonacci sequence up to and including the input position (e.g., the generated function would return "0, 1, 1, 2, 3" if "4" is the input). Therefore, to avoid this scenario, the user can provide test functions "assert.equal(x(0), 0)" and "assert.equal(x(1), 1)" that specify the desired output of the generated function given particular inputs.

[0041] If the PromptScript interface 114 determines that the returned code fails at any stage (e.g., it cannot be parsed, it fails a validation function, or it fails any of the test functions provided by the user), it essentially retries the code generation by repeating the request to the LLM to generate code until all of these evaluations pass.

[0042] After receiving code that passes all evaluations, the code may be returned to the programming language specific file 110 and / or application development client 104 for use within a computer program (e.g., for later compilation or interpretation) and / or written to the file system cache 118 for storage.

[0043] The PromptScript interface 114 allows scripting language commands written in PromptScript to be specified within computer code being written by a user. Thus, for example, if a user is creating a JavaScript program, the user can first indicate in the JavaScript program that the PromptScript package 112 should be loaded, such as by specifying a "const p = require('PromptScript')" command at or near the beginning of the JavaScript code. The PromptScript package then specifies a series of commands that can be used within the JavaScript code to trigger the LLM calls necessary to generate the code.

[0044] Thus, to continue with the Fibonacci function described above, a user can type the following into the software code of the software application 106:

[0045] Const result = p 'a fibonnaci function'.t(x => { assert.equal(x(0), 0) assert.equal(x(1), 1) }

[0046] From this, the PromptScript interface 114 may generate a natural language prompt as follows, which may then be passed to the LLM 116 along with the system message: Create your Javascript Fibonacci Function Next assert.equal(x(0), 0) assert.equal(x(1), 1) Something that satisfies

[0047] The system message may be as follows: const systemMsg = "You're a Node.js expert. Create the JavaScript thing I described and return it inside the ```js...``` fence so I can write `const result = eval(...)` and get it. Do not output any additional text. Example:\n```js\nconst x = function(a){ console.log(a)};x```" let userMsg = string if (testFunction) userMsg += '\nTherefore, it satisfies the following test function\n' + testFunction.toString()

[0048] 2 is a flow diagram illustrating a method 200 for automatically generating a portion of software programming code, according to an example embodiment. Here, in operation 202, a scripting language interface specified in the software programming code is invoked. The scripting language interface is designed to identify one or more specific commands in the software programming code. In operation 204, the scripting language interface identifies a command of the one or more specific commands in the software programming code. The command includes a first portion containing a first natural language prompt for generating the software code piece and, optionally, a second portion containing one or more test functions.

[0049] In operation 206, a hash is created from the first portion by passing the first portion through a hash function. In operation 208, the hash is used to look up the file system cache to determine whether it already contains the hash. If so, the piece of software code has already been generated and stored in the file system cache, and thus, in operation 210, the piece of software code is retrieved from the file system cache and inserted into the software programming code. More specifically, the code (in the form of a string) is evaluated using the eval function, so the code is essentially interpreted on the fly.

[0050] If not, in operation 212, the command (including the first portion and, if present, the second portion) is converted by the scripting language interface into a user portion of a second natural language prompt that also includes a predefined system message in the system portion. In operation 214, the second natural language prompt is sent as input to the LLM, which generates a piece of software code based on the second natural language prompt. In operation 216, the generated piece of software code is received by the scripting language interface.

[0051] Extraneous text is removed from the generated first piece of software code in operation 218. This may include removing any text outside of a markdown fence if a system message instructs the LLM to generate code within a markdown fence.

[0052] At operation 220, the generated software code piece is evaluated using an evaluation function for the programming language of the software programming code to determine whether the generated software code piece is valid according to one or more requirements of the programming language. If not valid, the method 200 returns to operation 214, where sending the second natural language prompt to the LLM is retried.

[0053] If so, then in operation 222, the generated software code piece is tested using one or more test functions, if present, to determine whether all of the one or more test functions evaluate to true. If not, then the method 200 returns to operation 214, where an attempt is made to send the second natural language prompt to the LLM again.

[0054] If evaluated to true, the generated piece of software code is inserted into the software programming code in operation 224. More specifically, the code (in the form of a string) is evaluated using the eval function, so that the code is essentially interpreted on the fly. In operation 226, the generated piece of software code is stored in a file system cache so that future invocations of the first prompt do not require the code to be regenerated, speeding up future invocations.

[0055] If an error is returned at any stage, the method 200 may cause a second natural language prompt to be resent to the LLM, which allows a different output to be generated by the LLM, at least in a non-zero temperature LLM (non-deterministic LLM). Optionally, any returned error may be provided to the LLM along with the second natural language prompt to effectively instruct the LLM to avoid that error in the future.

[0056] In light of the above disclosure, various examples are described below. It should be noted that one or more features of the examples taken separately or in combination should be considered within the disclosure of the present application.

[0057] Example 1. at least one hardware processor; and a computer-readable medium storing instructions, the instructions, when executed by the at least one hardware processor, causing the at least one hardware processor to: Identifying a command in the software programming code with a scripting language interface, the command being defined in a scripting language interface library; translating the command into a user portion of a natural language prompt; Adding predefined system parts to natural language prompts; sending natural language prompts to the LLM to generate a first piece of software code written in a first programming language based on the natural language prompts; verifying the first piece of software code using a verification function in a first programming language; and Inserting a first piece of software code into the software programming code A system that performs an operation including:

[0058] Example 2. The system of example 1, wherein the command includes a natural language portion that is a natural language description of a desired piece of programming code and one or more test functions.

[0059] Example 3. The action is, Executing one or more test functions on the first piece of software code Further comprising: the inserting is performed in response to determining that the first piece of software code passes all of the one or more test functions. The system in Example 2.

[0060] Example 4. The system of any of Examples 1-3, wherein in response to determining that the first piece of software code fails the validation function, the system regenerates the first piece of software code by resending the natural language prompt to the LLM.

[0061] Example 5. The system of example 3, wherein in response to determining that the first piece of software code failed one or more of the one or more test functions, the system regenerates the first piece of software code by resending the natural language prompt to the LLM.

[0062] Example 6. If the action is generating a hash of the command by applying a hash function to the command; storing the hash and the first piece of software code in a file system cache; The system of any of Examples 1-5, further comprising:

[0063] Example 7. The action is identifying a second command in the software programming code via the scripting language interface; generating a hash of the second command by applying a hash function to the command; determining whether a hash of the second command is contained within a file system cache; in response to determining that the hash of the second command is contained within the file system cache, retrieving the first piece of software code from the file system cache and inserting the first piece of software code into the software programming code rather than converting the command into a user portion of a second natural language prompt and sending the second natural language prompt to the LLM; The system of any of Examples 1-6, further comprising:

[0064] Example 8. identifying a command in the software programming code through a scripting language interface, the command being defined in a scripting language interface library; converting the command into a user portion of a natural language prompt; Adding predefined system parts to natural language prompts; and sending natural language prompts to the LLM to generate a first piece of software code in a first programming language based on the natural language prompts; verifying the first piece of software code using a verification function in a first programming language; Inserting a first piece of software code into the software programming code; A method comprising:

[0065] Example 9. The method of example 8, wherein the command includes a natural language portion that describes in natural language the desired piece of programming code and one or more test functions.

[0066] Example 10. Executing one or more test functions on the first piece of software code Further comprising: the inserting is performed in response to determining that the first piece of software code passes all of the one or more test functions. The method of Example 9.

[0067] Example 11. The method of any of Examples 8-10, regenerating the first piece of software code by resending the natural language prompt to the LLM in response to determining that the first piece of software code failed the validation function.

[0068] Example 12. The method of Example 11, regenerating the first piece of software code by resending the natural language prompt to the LLM in response to determining that the first piece of software code failed one or more of the one or more test functions.

[0069] Example 13. generating a hash of the command by applying a hash function to the command; storing the hash and the first piece of software code in a file system cache; The method of any of Examples 8-12, further comprising:

[0070] Example 14. identifying a second command in the software programming code via the scripting language interface; generating a hash of the second command by applying a hash function to the command; determining whether a hash of the second command is contained within a file system cache; in response to determining that the hash of the second command is contained within the file system cache, retrieving the first piece of software code from the file system cache and inserting the first piece of software code into the software programming code rather than converting the command into a user portion of a second natural language prompt and sending the second natural language prompt to the LLM; The method of any of Examples 8-13, further comprising:

[0071] Example 15. A non-transitory machine-readable medium storing instructions, the instructions, when executed by one or more processors, causing the one or more processors to: identifying a command in the software programming code through a scripting language interface, the command being defined in a scripting language interface library; converting the command into a user portion of a natural language prompt; Adding predefined system parts to natural language prompts; and sending natural language prompts to the LLM to generate a first piece of software code in a first programming language based on the natural language prompts; verifying the first piece of software code using a verification function in a first programming language; Inserting a first piece of software code into the software programming code; A non-transitory machine-readable medium for performing operations including:

[0072] Example 16. The non-transitory machine-readable medium of example 15, wherein the command includes a natural language portion that describes in natural language the desired piece of programming code and one or more test functions.

[0073] Example 17. The action is Executing one or more test functions on the first piece of software code Further comprising: the inserting is performed in response to determining that the first piece of software code passes all of the one or more test functions. The non-transitory machine-readable medium of Example 16.

[0074] Example 18. The non-transitory machine-readable medium of any of Examples 15-17, regenerating the first piece of software code by resending the natural language prompt to the LLM in response to determining that the first piece of software code failed the validation function.

[0075] Example 19. The non-transitory machine-readable medium of example 18, regenerating the first piece of software code by resending the natural language prompt to the LLM in response to determining that the first piece of software code failed one or more of the one or more test functions.

[0076] Example 20. The action is generating a hash of the command by applying a hash function to the command; storing the hash and the first piece of software code in a file system cache; 20. The non-transitory machine-readable medium of any of Examples 15-19, further comprising:

[0077] FIG. 3 is a block diagram 300 illustrating a software architecture 302 that may be installed on any one or more of the devices described above. It will be appreciated that FIG. 3 is merely a non-limiting example of a software architecture and that many other architectures may be implemented to enable the functionality described herein. In various embodiments, the software architecture 302 is implemented by hardware, such as machine 400 of FIG. 4, which includes a processor 410, memory 430, and input / output (I / O) components 450. In this exemplary architecture, the software architecture 302 may be conceptualized as a stack of layers, with each layer capable of providing a particular function. For example, the software architecture 302 includes layers such as an operating system 304, libraries 306, frameworks 308, and applications 310. In operation, the applications 310 invoke API calls 312 through this software stack and receive messages 314 in response to the API calls 312, consistent with certain embodiments.

[0078] In various implementations, the operating system 304 manages hardware resources and provides common services. The operating system 304 includes, for example, a kernel 320, services 322, and drivers 324. The kernel 320 serves as an abstraction layer between the hardware and other software layers, consistent with some embodiments. For example, the kernel 320 performs memory management, processor management (e.g., scheduling), component management, networking, and security configuration, among other functions. The services 322 may provide other common services for other software layers. The drivers 324 are responsible for controlling or interfacing with the underlying hardware, according to some embodiments. By way of example, the drivers 324 may include a display driver, a camera driver, a BLUETOOTH® or BLUETOOTH® Low-Energy driver, a flash memory driver, a serial communication driver (e.g., a Universal Serial Bus (USB) driver), a Wi-Fi® driver, an audio driver, a power management driver, and the like.

[0079] In some embodiments, the libraries 306 provide a low-level common infrastructure utilized by the applications 310. The libraries 306 can include system libraries 330 (e.g., the C standard library) that can provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. Additionally, the libraries 306 can include API libraries 332 such as a media library (e.g., a library for supporting the presentation and manipulation of various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework used for rendering in two dimensions (2D) and three dimensions (3D) in a graphical context on a display), a database library (e.g., SQLite for providing various relational database functions), a web library (e.g., WebKit for providing web browsing capabilities), etc. The libraries 306 may also include a wide variety of other libraries 334 for providing many other APIs to the applications 310 .

[0080] The framework 308 provides a high-level, common infrastructure that can be utilized by the applications 310, according to some embodiments. For example, the framework 308 provides various graphical user interface functions, high-level resource management, high-level location services, etc. The framework 308 can provide a wide range of other APIs that can be utilized by the applications 310, some of which may be specific to a particular operating system 304 or platform.

[0081] In an exemplary embodiment, the applications 310 include a wide variety of other applications, such as a home application 350, a contacts application 352, a browser application 354, a book reader application 356, a location application 358, a media application 360, a messaging application 362, a game application 364, and a third party application 366. According to some embodiments, the applications 310 are programs that perform functions defined within the program. One or more of the applications 310 may be structured in a variety of ways using a variety of programming languages, such as object-oriented programming languages ​​(e.g., Objective-C, Java, or C++) or procedural programming languages ​​(e.g., C or assembly language). In one example, the third party application 366 (e.g., an application developed using the ANDROID™ or IOS™ Software Development Kit (SDK) by an entity other than the vendor of a particular platform) may be mobile software that runs on a mobile operating system, such as IOS™, ANDROID™, WINDOWS™ Phone, or another mobile operating system. In this example, the third party application 366 can invoke API calls 312 provided by the operating system 304 to enable the functionality described herein to be implemented.

[0082] FIG. 4 illustrates a schematic diagram of a machine 400 in the form of a computer system, in which a set of instructions may be executed to cause the machine 400 to perform any one or more of the methods discussed herein, according to an exemplary embodiment. In particular, FIG. 4 illustrates a schematic diagram of a machine 400 in the form of an exemplary computer system, in which instructions 416 (e.g., software, programs, applications, applets, apps, or other executable code) may be executed to cause the machine 400 to perform any one or more of the methods discussed herein. For example, the instructions 416 may cause the machine 400 to perform the method 200 of FIG. 2. Additionally or alternatively, the instructions 416 may also implement FIGS. 1-2, etc. The instructions 416 transform a general unprogrammed machine 400 into a specific machine 400 programmed to perform the functions described and illustrated in the manner described. In alternative embodiments, machine 400 may operate as a stand-alone device or machine 400 may be coupled (e.g., networked) to other machines. In a networked deployment, machine 400 may operate as a server or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.The machine 400 may comprise, without limitation, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smartphone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing instructions 416, sequentially or otherwise, that specify actions to be taken by the machine 400. Additionally, although only a single machine 400 is illustrated, the term "machine" shall be taken to include a collection of machines 400 that individually or jointly execute instructions 416 to perform any one or more of the methodologies discussed herein.

[0083] Machine 400 may include a processor 410, memory 430, and I / O components 450, which may be configured to communicate with each other, such as via a bus 402. In an exemplary embodiment, processor 410 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 412 and a processor 414 capable of executing instructions 416. The term "processor" is intended to include a multi-core processor, which may include two or more independent processors (sometimes referred to as "cores") capable of simultaneously executing instructions 416. Although FIG. 4 shows multiple processors 410, the machine 400 may include a single processor with a single core 412, a single processor with multiple cores 412 (e.g., a multi-core processor 412), multiple processors with single cores 412, 414, multiple processors with multiple cores 412, 414, or any combination thereof.

[0084] Memory 430 may include a main memory 432, a static memory 434, and a storage unit 436, each accessible to processor 410, such as via bus 402. Main memory 432, static memory 434, and storage unit 436 store instructions 416 embodying any one or more of the methods or functions described herein. Instructions 416 may reside, in whole or in part, in main memory 432, in static memory 434, in storage unit 436, within at least one of processors 410 (e.g., within a processor's cache memory), or any suitable combination thereof, during their execution by machine 400.

[0085] The I / O components 450 can include a wide variety of components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, and the like. The particular I / O components 450 included within a particular machine will vary depending on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It will be appreciated that the I / O components 450 may include many other components not shown in FIG. 4. The I / O components 450 are grouped according to function merely to simplify the following discussion, and such grouping is in no way limiting. In various exemplary embodiments, the I / O components 450 can include an output component 452 and an input component 454. Output components 452 can include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., a speaker), haptic components (e.g., vibration motors, resistive mechanisms), other signal generators, etc. Input components 454 can include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input component), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing device), tactile input components (e.g., a physical button, a touch screen that indicates the location and / or force of a touch or touch gesture, or other tactile input component), audio input components (e.g., a microphone), etc.

[0086] In further exemplary embodiments, the I / O component 450 can include a biometric component 456, a motion component 458, an environmental component 460, or a position component 462, among many other components. For example, the biometric component 456 can include components for detecting expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying a person (e.g., voice identification, retina identification, face identification, fingerprint identification, or electroencephalogram-based identification), etc. The motion component 458 can include an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. The environmental components 460 may include, for example, a light sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers to detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones to detect background noise), a proximity sensor component (e.g., an infrared sensor to detect nearby objects), a gas sensor (e.g., a gas detection sensor to detect concentrations of harmful gases for safety or to measure pollutants in the air), or other components capable of providing indications, measurements, or signals corresponding to the surrounding physical environment. The position components 462 may include a location sensor component (e.g., a global positioning system (GPS) receiver component), an altitude sensor component (e.g., an altimeter or a barometer to detect air pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), etc.

[0087] Communication can be implemented using a wide variety of technologies. I / O component 450 can include a communication component 464 operable to couple machine 400 to network 480 or device 470 via coupling 482 and coupling 472, respectively. For example, communication component 464 can include a network interface component or another device suitable for interfacing with network 480. In further examples, communication component 464 can include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, a Bluetooth component (e.g., Bluetooth Low Energy), a Wi-Fi component, and other communication components for communicating via other modalities. Device 470 can be another machine or any of a wide variety of peripheral devices (e.g., coupled via USB).

[0088] Further, the communication component 464 may detect the identifier or may include a component operable to detect the identifier. For example, the communication component 464 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as QR codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying tagged audio signals). In addition, a variety of information may be obtained via the communication component 464, such as location via Internet Protocol (IP) geolocation, location via triangulation with Wi-Fi signals, location via detection of NFC beacon signals that may indicate a particular location.

[0089] The various memories (e.g., 430, 432, 434, and / or memory of processor 410) and / or storage unit 436 may store one or more sets of instructions 416 and data structures (e.g., software) that embody or are utilized by any one or more of the methods or functions described herein. These instructions (e.g., instructions 416), when executed by processor 410, cause various operations to be performed for implementing the disclosed embodiments.

[0090] As used herein, the terms "machine storage medium," "device storage medium," and "computer storage medium" mean the same thing and may be used interchangeably. These terms refer to a single or multiple storage devices and / or storage media (e.g., a centralized or distributed database, and / or associated caches and servers) that store executable instructions and / or data. Thus, these terms shall be interpreted to include, but are not limited to, solid-state memory and optical and magnetic media, including memory that is internal or external to the processor. Specific examples of machine storage media, computer storage media, and / or device storage media include, by way of example, semiconductor memory devices, such as non-volatile memory including erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), field programmable gate arrays (FPGAs), and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage media," "computer storage media," and "device storage media" specifically exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal media" discussed below.

[0091] In various exemplary embodiments, one or more portions of network 480 may be an ad-hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi network, another type of network, or a combination of two or more such networks. For example, network 480 or a portion of network 480 may include a wireless or cellular network, and coupling 482 may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or another type of cellular or wireless coupling. In this example, coupling 482 may implement any of a variety of types of data transfer technologies, such as single carrier radio transmission technology (1xRTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data Rates for GSM Evolution (EDGE) technology, 3G networks, Third Generation Partnership Project (3GPP) including Fourth Generation Wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standards, others defined by various standards bodies, other long-range protocols, or other data transfer technologies.

[0092] The instructions 416 may be transmitted or received over the network 480 using a transmission medium via a network interface device (e.g., a network interface component included in the communications component 464) and utilizing any one of several well-known transfer protocols (e.g., HTTP). Similarly, the instructions 416 may be transmitted or received using a transmission medium to the device 470 via a connection 472 (e.g., a peer-to-peer connection). The terms "transmission medium" and "signal medium" mean the same thing, and they may be used interchangeably in this disclosure. The terms "transmission medium" and "signal medium" shall be interpreted to include any intangible medium capable of storing, encoding, or carrying the instructions 416 so that they may be executed by the machine 400, as well as to include digital or analog communications signals, or other intangible media for facilitating the communication of such software. Thus, the terms "transmission medium" and "signal medium" shall be interpreted to include any form of modulated data signal, carrier wave, or the like. The term "modulated data signal" means a signal having one or more of its characteristics set or changed to encode information within the signal.

[0093] The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure. These terms are defined to include both machine storage media and transmission media. Thus, these terms include both storage devices / storage media and carrier waves / modulated data signals. [Explanation of symbols]

[0094] 100 Systems 102 Application Development Server 104 Application Development Client 106 Software Applications 108 Library 110 Programming Language Specific Files 112 PromptScript Package 114 PromptScript Interface 116 LLM 118 File System Cache 200 ways 300 Block Diagram 302 Software Architecture 304 Operating Systems 306 Library 308 Framework 310 Applications 312 API Calls 314 Messages 320 Kernel 322 Service 324 Driver 330 System Library 332 API Library 334 Other Libraries 350 Home Applications 352 Contact Application 354 Browser Applications 356 Book Reader Application 358 Location Application 360 Media Applications 362 Messaging Applications 364 Game Application 366 Third Party Applications 400 Machines 402 Bus 410 Processor 412 processor, multi-core processor 414 processor 416 Command 430 Memory 432 Main memory 434 Static Memory 436 Storage Unit 450 Input / Output (I / O) Components 452 Output Component 454 Input Component 456 Biometric Components 458 Movement Components 460 Environmental Components 462 Position Component 464 Communication Components 470 devices 472 Combine 480 Network 482 Combine

Claims

1. at least one hardware processor; and a computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to: Identifying a command in the software programming code with a scripting language interface, the command being defined in a scripting language interface library; converting said command into a user portion of a natural language prompt; adding predefined system portions to said natural language prompts; submitting the natural language prompts to a large language model (LLM) to generate a first piece of software code in a first programming language based on the natural language prompts; verifying the first piece of software code using a verification function in the first programming language; and Inserting the first piece of software code into the software programming code. A system that performs an operation including:

2. The system of claim 1 , wherein the command includes a natural language portion that describes in natural language a desired piece of programming code and one or more test functions.

3. The operation, executing the one or more test functions on the first piece of software code; Further comprising: the inserting is performed in response to determining that the first piece of software code passes all of the one or more test functions. The system of claim 2.

4. 2. The system of claim 1, further comprising: in response to determining that the first piece of software code fails the verification function, regenerating the first piece of software code by resending the natural language prompt to the LLM.

5. 4. The system of claim 3, further comprising: in response to determining that the first piece of software code fails one or more of the one or more test functions, regenerating the first piece of software code by resending the natural language prompt to the LLM.

6. The operation, generating a hash of the command by applying a hash function to the command; storing the hash and the first piece of software code in a file system cache; The system of claim 1 further comprising:

7. The operation, identifying a second command in the software programming code via the scripting language interface; generating a hash of the second command by applying the hash function to the command; determining whether the hash of the second command is contained within the file system cache; in response to determining that the hash of the second command is contained within the file system cache, retrieving the first piece of software code from the file system cache and inserting the first piece of software code into the software programming code rather than converting the command into a user portion of a second natural language prompt and sending the second natural language prompt to the LLM; The system of claim 6, further comprising:

8. identifying a command in the software programming code by a scripting language interface, the command being defined in a scripting language interface library; translating said command into a user portion of a natural language prompt; adding predefined system portions to the natural language prompt; sending the natural language prompts to a large language model (LLM) to generate a first piece of software code in a first programming language based on the natural language prompts; verifying the first piece of software code using a verification function in the first programming language; inserting the first piece of software code into the software programming code; A method comprising:

9. 9. The method of claim 8, wherein the command includes a natural language portion that describes in natural language the desired piece of programming code and one or more test functions.

10. executing the one or more test functions on the first piece of software code. Further comprising: the inserting step is performed in response to determining that the first piece of software code passes all of the one or more test functions. The method of claim 9.

11. 9. The method of claim 8, further comprising: regenerating the first piece of software code by resending the natural language prompt to the LLM in response to determining that the first piece of software code fails the verification function.

12. 11. The method of claim 10, further comprising: regenerating the first piece of software code by resending the natural language prompt to the LLM in response to determining that the first piece of software code fails one or more of the one or more test functions.

13. generating a hash of the command by applying a hash function to the command; storing the hash and the first piece of software code in a file system cache; 9. The method of claim 8, further comprising:

14. identifying, via the scripting language interface, a second command in the software programming code; generating a hash of the second command by applying the hash function to the command; determining whether the hash of the second command is contained within the file system cache; in response to determining that the hash of the second command is contained within the file system cache, retrieving the first piece of software code from the file system cache and inserting the first piece of software code into the software programming code rather than converting the command into a user portion of a second natural language prompt and sending the second natural language prompt to the LLM; 14. The method of claim 13, further comprising:

15. A non-transitory machine-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to: identifying a command in the software programming code through a scripting language interface, the command being defined in a scripting language interface library; translating said command into a user portion of a natural language prompt; adding predefined system portions to said natural language prompts; submitting the natural language prompts to a large language model (LLM) to generate a first piece of software code in a first programming language based on the natural language prompts; verifying the first piece of software code using a verification function in the first programming language; inserting the first piece of software code into the software programming code; A non-transitory machine-readable medium for causing operations to be performed, including:

16. 16. The non-transitory machine-readable medium of claim 15, wherein the commands include a natural language portion that describes in natural language a desired piece of programming code and one or more test functions.

17. The operation, executing the one or more test functions on the first piece of software code; Further comprising: the inserting is performed in response to determining that the first piece of software code passes all of the one or more test functions.

17. The non-transitory machine-readable medium of claim 16.

18. 16. The non-transitory machine-readable medium of claim 15, wherein in response to determining that the first piece of software code fails the verification function, the first piece of software code is regenerated by resending the natural language prompt to the LLM.

19. 20. The non-transitory machine-readable medium of claim 17, wherein in response to determining that the first piece of software code fails one or more of the one or more test functions, the first piece of software code is regenerated by resending the natural language prompt to the LLM.

20. The operation, generating a hash of the command by applying a hash function to the command; storing the hash and the first piece of software code in a file system cache; 20. The non-transitory machine-readable medium of claim 15, further comprising:

Images