Emissible text for cue designs

Abstract

An omissible text is constructed that prioritizes content included in hints of a large language model having a context window of a fixed size. Generation of texts according to instructions generated by developers or automatic generation of source codes within a source code editor may be omitted. The source code editor may include features that select certain code lines as important or focal, which are assigned high priority values. The changed line, the source code line at the current cursor position, the source code line at the beginning of the file, and the line of output data are regarded as focus lines. The non-focal row is assigned a priority based on a distance from the focal row. Text may be omitted to limit data included in the hint to a contextual window size by replacing the lowest value text and source code line with a replacement string.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of an earlier filed provisional application, serial number 63 / 452,664, filed on March 16, 2023, entitled “Elidable Text For PromptCrafting,” which is incorporated herein by reference in its entirety. Background Art

[0003] A large language model is a machine learning model that is trained on massive training datasets of text and / or source code, resulting in models containing billions of parameters. Large language models are used for a variety of tasks, such as natural language processing, text generation, machine translation, and source code generation. Large language models are based on deep learning neural networks, such as the Neural Transformer model with attention.

[0004] Large language models are given a prompt, which is a sequence of tokenized inputs, which the model processes to make a prediction or output. A context window is a collection of tokens that the model can access and use in its predictions. The size of the context window dictates the number of tokens that can be used in a prompt. Limited context windows can sometimes become challenging when the model requires a significant amount of background information to make a prediction beyond the size of the context window. Summary of the Invention

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0006] Omittable text is used to prioritize the content included in hints for large language models to fit within a fixed-size context window and to assemble shortened hints based on priority. Omittable text is generated based on developer-generated instructions or automatically generated for source code such as within a source code editor. A source code editor or other code processing system may select certain lines of code as important or focal features, which are assigned high priority values. Non-focus lines of code are assigned priority based on their semantic distance from the focal line. Lines that are changed, source code lines at the beginning of a file, and lines that output data are considered focal lines. Omittable text limits the data included in the hint to the size of the context window by replacing the lowest-valued text and source code lines with a replacement string.

[0007] These and other features and advantages will be apparent from a reading of the following detailed description and a review of the associated drawings.It is to be understood that both the foregoing general description and the following detailed description are merely illustrative and not restrictive of the aspects, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 An exemplary system is shown for creating ellipsisable text and generating prompts for a large language model based on developer-generated instructions.

[0009] Figure 2 An exemplary system for automatically creating ellipsisable text based on user actions in a source code editor is shown.

[0010] Figure 3 is a flow chart illustrating an exemplary method for creating a system for omitting text and prompts.

[0011] Figure 4 is a flow chart of an exemplary method for generating prompts within a fixed-size context window of a large language model using ellipsisable text.

[0012] Figure 5 is a block diagram illustrating an exemplary operating environment. DETAILED DESCRIPTION

[0013] Overview

[0014] The subject matter of the present disclosure relates to an automated system for generating ellipsis text and using ellipsis text to constrain prompts for large language models to fit within the size of the large language model's context window. The ellipsis text prioritizes text and source code lines using a priority value that indicates the importance of each line in the prompt. In one aspect, the value can range from 0 to 1, with 1 being the highest value and 0 being the lowest. The ellipsis text is used to replace the lowest-valued text and source code lines with a replacement string, such as a bracketed ellipsis […].

[0015] Omittable text can be generated by a developer. In one aspect, a developer can explicitly generate an ellipsis class. The input for creating the class can include a data structure (e.g., a constant, an array, etc.) that includes prioritized text and source code lines or blocks, and optionally other previously constructed ellipsis texts, each with a priority value. The ellipsis class can contain functions that perform reduction analysis on the input data using the prioritized list.

[0016] In one aspect, ellipsis text can be automatically created without the involvement of a developer. The ellipsis text generator interacts with a source code editor to select certain lines in the source code as focus lines that are assigned high priority values. Based on previously set options, the focus lines include the source code line at the current cursor position, the code line that is changed, the source code line at the beginning of the program, and the code line that outputs data. The ellipsis text generator assigns priorities to the remaining source code lines based on a distance metric from the focus line. The source code editor can utilize a code completion feature that generates code completion candidates to complete partially formed source code snippets. The hint design engine can then generate hints for a large language model to use the ellipsis text to generate code completion candidates to prioritize the text and source code lines included in the hint.

[0017] Attention is now directed to systems, apparatus, and methods for designing systems that can omit text prompts.

[0018] system

[0019] Figure 1 1 shows an exemplary system 100 for creating prompts using omittable text based on developer-provided instructions. System 100 includes an omittable text generator 102, a prompt design engine 104, and a large language model 106. Omittable text generator 102 receives instructions 108 specifying how to generate omittable text. Prompt design engine 104 uses omittable text 112 to generate a prompt 116, which is applied to large language model 106 to predict a response 118.

[0020] In one aspect, the prompt 116 comprises a question, and the large language model 106 responds by predicting a response to the question. In one aspect, the question comprises instructions for a task for the model to perform and a context comprising source code for performing the task.

[0021] In one aspect, developer-generated instructions can be implemented as a class with constants 126 describing the priority order of data to be included in prompts 128-138 and a method 140 for generating prompts. Block 120 illustrates an exemplary construction of an object-oriented class, question, written in the TypeScript programming language, that contains editable text consisting of six lines as constants 126 and a makePrompt method 140. The makePrompt method 140 includes a parameter indicating the length of the prompt, which is the maximum number of tokens included in the prompt.

[0022] The construction of the omissible text 128-138 includes a text string 128 with a value of 0.5, a text string 130 with a value of 0.9, a diff file 132 (this.diff) with a value of 0.7, a text string 134 with a value of 0.95, a source code file 136 with a value of 0.8, and a text string 138 with a value of 1.

[0023] The omissible text generator 102 converts the developer-provided instructions into prioritized text and omissible text objects 122 of source code lines 142-166. For example, lines 146-154 are lines from the file this.diff. Each line from this.diff is included in the omissible text 122 with a corresponding priority value. Lines 160-164 include source code lines from the source code file documentWithLanguage, each with a corresponding priority value.

[0024] The prompt design engine 104 generates a prompt 124 using the ellipsis text 122. The data in the prompt is sorted in the same order as listed in the ellipsis text 122. The eliminated lines replaced with bracketed ellipsis […] include line 142, lines 146-148, and line 154. The bracketed ellipsis is indented at the same spacing as the eliminated lines.

[0025] In one aspect, the large language model 106 is a neural transformer model with attention. The neural transformer model with attention is a unique type of machine learning model. Machine learning involves the use and development of computer systems that can learn and adapt without following explicit instructions by analyzing patterns in data using algorithms and statistical models and drawing inferences from them. Machine learning uses different types of statistical methods to learn from data and predict future decisions. Traditional machine learning includes classification models, data mining, Bayesian networks, Markov models, clustering, and visual data mapping.

[0026] Deep learning differs from traditional machine learning in that it uses multi-stage data processing through multiple hidden layers of a neural network to learn and interpret features and the relationships between features. Deep learning embodies neural networks, which is different from traditional machine learning techniques that do not use neural networks. The Neural Transformer model is a type of deep learning that utilizes an attention mechanism. Attention guides the neural network to focus on a subset of features or tokens in the input sequence, thereby learning different representations from different positions of the tokens in the input sequence. The Neural Transformer model uses attention to handle the dependency between its input and output without using recurrent neural networks (RNNs) (such as long short-term memory (LSTM) networks) and convolutional neural networks (CNNs).

[0027] There are various configurations of neural transformer models with attention. In one aspect, a large language model can be configured as an encoder-decoder neural transformer model with attention or a decoder-only neural transformer model with attention. An encoder-decoder neural transformer model with attention consists of a series of stacked encoder blocks coupled to a series of stacked decoder blocks. The encoder blocks extract features from the input sequence, generate embeddings for each token in the input sequence, and the decoder uses these features to produce an output sentence. A decoder neural transformer model with attention includes a series of stacked decoder blocks.

[0028] Large language models are pre-trained on natural language text and source code. Examples of neural transformer models with attention include OpenAI's GPT model and OpenAI's Codex model. In one aspect, large language models are hosted on external servers and accessed over the network via an application programming interface (API). Training large language models requires a considerable amount of training data and computing resources, which prevents some developers from creating their own models. Instead, third parties that have already created large language models can provide them to developers as cloud services for a fee or for free.

[0029] The input or prompt to a large language model is limited by the size of its context window. The context window is the number of tokens that the large language model can process in a forward pass. These tokens represent the natural language text and source code contained in the prompt. Each token corresponds to a small set of characters. The size of the context window varies and can include 4096 tokens (such as for Github's CoPilot Codex) to 32,000 tokens (for OpenAI's GPT-4 model).

[0030] Figure 2 An exemplary system 200 for creating hints using ellipsis text generated based on user actions while in a source code editor is shown. System 200 includes a source code editor 202, an ellipsis text generator 206, a hint design engine 210, and a large language model 214. The ellipsis text generator 206 identifies a focused line 204 in a source code program that is considered important, such as the line where the cursor is currently located in the editor or a line that has been changed since the document was last saved. Alternatively, the ellipsis text generator 206 can use a default setting that considers lines at the beginning and end of a file important.

[0031] Hint design engine 210 uses ellipsis text 208 to generate hints 212, which are applied to large language model 214 to predict responses, such as code completion candidates 216. Hints can include the context of the source code snippet being developed in the source code editor. The ellipsis text is used to limit the size of the hint to the context window size of the large language model.

[0032] method

[0033] Disclosed herein are descriptions of various exemplary methods utilizing the systems and devices. The operation of various aspects may be further described with reference to the various exemplary methods. It will be understood that, unless otherwise indicated, the representative methods need not be performed in the order presented or in any particular order. Furthermore, the various activities described with respect to these methods may be performed in a serial or parallel manner, or in any combination of serial and parallel operations. In one or more aspects, the methods illustrate the operation of the systems and devices disclosed herein.

[0034] Figure 3 An exemplary method 300 for a hint design system is shown. Omittable text can be generated by a developer or automatically generated based on code changes or code edits using default settings (block 302). The hint design engine uses the omittable text to generate hints within the context window size by replacing text and source code lines with replacement strings that have low priority values ​​(block 304). The hints are then sent to a large language model, which generates a response (block 306), which is output by the target system (block 308).

[0035] In one aspect, elliptical text can be generated based on developer-generated instructions (block 302). The developer-generated instructions can call a software package with instructions and files for generating prompts. These instructions and files constitute a block of text, code, or a previously constructed list of ElidableTexts, optionally with a priority value or a line of code designated as a focus line. The package can include a class ElidableText that processes its input when instantiated to generate an object ElidableText that contains text and source code lines with corresponding priority values. The text and source code lines are listed in the order in which they are intended to appear in the prompt. The priority value guides the selection of which text or source code lines to be included in the prompt. The package can include a function makePrompt that generates a prompt when executed. In this scenario, the makePrompt function is the prompt design engine.

[0036] In another aspect, ellipsis text can be automatically generated (302) for source code in a source code editor or other source code processing tool. A background process of the source code editor converts the source code into an indentation tree. The indentation tree shows the hierarchical relationship between different code elements. Indentation is a collection of spaces added to the beginning of a code statement or code block. Source code statements within the same indentation belong to the same code block. The number of spaces in an indentation represents the indentation level. Different indentation levels represent relationships between source code statements, such as parent, child, and sibling.

[0037] In one aspect, the developer can select one or several source code statements as the focus line, or the text generator can be omitted to select the line in the current cursor position as the focus line. For each non-focus line, a distance metric is used to calculate the distance from the non-focus line to the focus line. The distance metric is used to set the priority value of the line. Lines that are farther away from the focus line are given lower priority values, while lines that are closer to the focus line are given higher priority values. In one aspect, the term "distance" refers to the tree distance or the total weight of the shortest path to the focus line, where traversal from parent to child, traversal from child to parent, and traversal between siblings all have pre-configured weights.

[0038] Additionally, if the source code file has been changed from a previous version of the file, the changed line is considered the focus line (302). The changes can be displayed in the source code editor in a diff format, where added lines are preceded by a "+" character, and lines deleted from the file are preceded by a "-" character. Lines that have been added are considered the focus line and are given a higher priority value, while unchanged lines are given a lower priority value.

[0039] Alternatively, a default setting may be used (block 302). In this case, important code is considered to be located at the beginning of the file (such as import statements and leading comments) and statements at the end of the file (such as data output). The source code is parsed into an indented tree, and the first line of the indented tree is typically an import statement and / or leading comment, which are given a high priority value. The leaves of the indented tree typically contain individual statements, rather than statement blocks. Some of these lines may produce output that is given a high priority value.

[0040] Figure 4 4 is an exemplary method 400 for generating a prompt using ellipsisable text to reduce the size of the prompt so as to fit within a context window. A prompt design engine receives the ellipsisable text and a target markup length for the prompt (block 402). In one aspect, the prompt design engine can be a makePrompt function of a developer-generated class.

[0041] The hint design engine tokenizes each text and source code line of the ellipsisable text to determine the current token length (block 404). If the current token length is greater than the target token length for the context window (block 406), the ellipsisable text is used to replace portions of the text and / or source code with replacement strings (such as bracketed ellipsis) until the target token length is met (block 408). The method begins by finding the lowest-valued line (block 410). If the current lowest-valued line is not adjacent to another previously replaced line, the lowest-valued line is replaced with the replacement string and the lowest-valued line is located using the replacement line's indent spacing (block 412).

[0042] If the current lowest-valued line is adjacent to a single previously replaced line, the current lowest-valued line and the adjacent replacement string are replaced with a single bracketed ellipsis (block 414). The bracketed ellipsis is positioned with a minimum number of indent spacings based on the position of the current lowest-valued line and the position of the adjacent bracketed ellipsis (block 414).

[0043] If the current lowest-valued row is adjacent to two previously replaced rows, the current lowest-valued row and the adjacent replacement string are replaced with a single bracketed ellipsis (block 416). The new bracketed ellipsis is positioned with the minimum number of indent spacings from the position of the replacement string and the current lowest-valued row (block 416).

[0044] The hint design engine then recalculates the current tag length by subtracting the tag of the current lowest-valued line replaced by the replacement string and the potential tag length savings from the shortened indentation, and adding the cost of any inserted ellipsis lines (block 418). If the target tag length is not met, the process (blocks 406-418) is repeated until the target tag length is reached.

[0045] Once the target token length is reached, prompts are created in the order presented in the ellipsis text (block 420). Figure 1 As shown in the prompt 124 in , the omissible text lists the lines in the order in which they are intended to appear in the prompt. Lines 142, 146-148, 154 have been replaced with replacement strings, and the order in which the omissible text 122 was presented is maintained in the prompt.

[0046] Exemplary Operating Environment

[0047] Attention now turns to a discussion of an exemplary operating environment. Figure 5An exemplary operating environment 500 is shown in which one or more computing devices 502 are used to generate ellipsis text and prompts, and one or more computing devices 504 are used to host a large language model that uses the prompts to predict responses. However, it should be noted that the aspects disclosed herein are not limited to any particular configuration of devices. In another aspect, a single computing device can host the large language model and generate ellipsis text and prompts.

[0048] Computing devices 502 and 504 can be any type of electronic device, such as, but not limited to, a mobile device, a personal digital assistant, a mobile computing device, a smartphone, a cellular phone, a handheld computer, a server, a server array or server farm, a web server, a network server, a blade server, an internet server, a workstation, a minicomputer, a mainframe computer, a supercomputer, a network appliance, a web appliance, a distributed computing system, a multiprocessor system, or a combination thereof. Operating environment 500 can be configured in a network environment, a distributed environment, a multiprocessor environment, or a standalone computing device capable of accessing remote or local storage devices.

[0049] Computing devices 502, 504 may include one or more processors 508, 530, one or more communication interfaces 510, 532, one or more storage devices 512, 536, one or more input / output devices 514, 534, and one or more memory devices 516, 538. Processors 508, 530 may be any commercially available or custom processor and may include dual microprocessor and multi-processor architectures. Communication interfaces 510, 532 facilitate wired or wireless communication between computing devices 502, 504 and other devices. Storage devices 512, 536 may be computer-readable media that do not contain propagated signals, such as modulated data signals transmitted via carrier waves. Examples of storage devices 512, 536 include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile discs (DVDs) or other optical storage, magnetic cassettes, tape, and disk storage, all of which do not contain propagated signals, such as modulated data signals transmitted via carrier waves. There may be multiple storage devices 512, 536 in computing devices 502, 504. The input / output devices 514 , 534 may include a keyboard, a mouse, a pen, a voice input device, a touch input device, a display, a speaker, a printer, the like, and any combination thereof.

[0050] The memory device or memory 516, 538 can be any non-transitory computer-readable storage medium that can store executable processes, applications, and data. The computer-readable storage medium does not involve propagating signals, such as modulated data signals transmitted via a carrier wave. It can be any type of non-transitory storage device (e.g., random access memory, read-only memory, etc.), magnetic storage, volatile storage, non-volatile storage, optical storage, DVD, CD, floppy disk drive, etc., which does not involve propagating signals, such as modulated data signals transmitted via a carrier wave. The memory devices 516, 538 can also include one or more external storage devices or remotely located storage devices, which do not involve propagating signals, such as modulated data signals transmitted via a carrier wave.

[0051] Memory devices 516 and 538 may contain instructions, components, and data. A component is a software program that performs a specific function and is also referred to as a module, program, component, and / or application. Memory device 516 may include an operating system 518, an ellipsis generator 520, a hint design engine 522, ellipsis 524, developer-provided instructions 526, a source code editor 528, and other applications and data 528. Memory device 538 may include an operating system 540, a large language model 542, and other applications and data 544.

[0052] The computing device 502 may be communicatively coupled via a network 506. The network 506 may be configured as an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of a public switched telephone network (PSTN), a plain old telephone service (POTS) network, a wireless network, network, or any other type of network or combination of networks.

[0053] The network 506 may employ various wired and / or wireless communication protocols and / or technologies. The generations of different communication protocols and / or technologies that may be employed by the network may include, but are not limited to, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access 2000 (CDMA-2000), High Speed ​​Downlink Packet Access (HSDPA), Long Term Evolution (LTE), Universal Mobile Telecommunications System (UMTS), Evolution-Data Optimized (EV-DO), Worldwide Interoperability for Microwave Access (WiMax), Time Division Multiple Access (TDMA), Orthogonal Frequency Division Multiplexing (OFDM), Ultra-Wideband (UWB), Wireless Application Protocol (WAP), User Datagram Protocol (UDP), Transmission Control Protocol / Internet Protocol (TCP / IP), any portion of the Open Systems Interconnection (OSI) model protocols, Session Initiation Protocol / Real-time Transport Protocol (SIP / RTP), Short Message Service (SMS), Multimedia Messaging Service (MMS), or any other communication protocols and / or technologies.

[0054] Technical Effects

[0055] Aspects of the subject matter disclosed herein relate to the technical problem of designing prompts that fit within the context window size of a large language model. A technical feature associated with solving this problem is ellipsis text that specifies the priority of data used in the prompt. The technical effect achieved is constructing prompts within the context window size using the context that is most important for the model's predictions, without unduly increasing the computational burden.

[0056] Those skilled in the art understand that technical effects are the purpose of technical embodiments. The fact that an embodiment involves computation does not, in itself, remove the existence of a technical effect, nor does it change the specific and technical nature of the embodiment. The operations used to create ellipsis text and create prompts are understood herein to be digital in nature. The human mind cannot directly interface with a CPU or network interface card or other processor, or with RAM or other digital storage, to read or write the necessary data and perform the necessary operations on digital values ​​in the manner disclosed herein.

[0057] The embodiments are also assumed to be capable of running at scale within strict time constraints in a production environment or in a test lab within a production environment, and are not just thought experiments.

[0058] in conclusion

[0059] A system is disclosed, comprising: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for performing the following actions: accessing a large language model with a fixed-size context window, wherein the large language model generates predictions for a task given a prompt; receiving omissible text, wherein the omissible text includes an ordered data list, wherein the ordered data list includes text and source code statement lines with associated priority values; determining the size of the ordered data list; when the size of the ordered data list exceeds the fixed size of the context window: selecting an item in the ordered data list to exclude from the ordered data list based on the priority value associated with each line; and replacing the selected item from the ordered data list with a first replacement string; and constructing a prompt for the large language model based on the ordered list that satisfies the fixed size of the context window.

[0060] In one aspect, the one or more programs include further instructions for submitting a prompt to the large language model and receiving a response corresponding to the prompt from the large language model. In one aspect, the one or more programs include further instructions for concatenating the first replacement string with an existing replacement string adjacent to an item excluded from the ordered data list.

[0061] In one aspect, the one or more programs include instructions for creating ellipsisable text based on a source code program, wherein import statements and statements that output data from the source code program have a high priority value. In one aspect, the one or more programs include instructions for creating ellipsisable text based on a focus line of the source code program in a source code editor, wherein the focus line is associated with the high priority value, wherein the focus line comprises a source code line that has been changed or a source code line at a current cursor position in the source code editor.

[0062] In one aspect, the one or more programs include instructions to associate a priority value with a non-focused line of a source code program based on a distance from the non-focused line to the focused line. In one aspect, the large language model is a neural transformer model with attention.

[0063] A computer-implemented method is disclosed, comprising: accessing a large language model that accepts hints within a context window size; obtaining omissible text comprising text lines and source code statement lines, wherein each of the text lines and source code statement lines comprises a priority value, wherein the large language model is associated with the context window; determining the sizes of the text lines and source code statement lines; and when the sizes of the text lines and source code statement lines exceed the size of the context window: selecting a line of the text lines and a line of the source code statement lines based on a lowest priority value; replacing the selected one of the text lines and the selected one of the source code statement lines with a replacement string in priority order; and constructing a hint to the large language model using the data of the omissible text in priority order.

[0064] In one aspect, the computer-implemented method further comprises creating the omissible text based on developer-generated instructions, wherein the developer-generated instructions comprise a class having a first data element and a method for generating a prompt having the omissible text, wherein the first data element comprises a sequence of lines of text and lines of source code statements.

[0065] In one aspect, the computer-implemented method further comprises: combining each replacement string adjacent to a replacement item in the ellipsis text into a single replacement string. In one aspect, the computer-implemented method further comprises: maintaining indentation of source code lines within the hint. In one aspect, the computer-implemented method further comprises: creating the ellipsis text based on a focused line of a source code program, wherein the focused line is a line of source code that is being altered, wherein the focused line is associated with a high priority value; and associating priority values ​​for non-focused lines based on their distance from the focused line.

[0066] In one aspect, the computer-implemented method further includes: creating the ellipsisable text based on a focus line of a source code program, wherein the focus line is a source code line at a current cursor position in a source code editor, wherein the focus line is associated with a high priority value; and associating priority values ​​for non-focus lines based on a distance from the focus line.

[0067] In one aspect, the computer-implemented method further comprises: creating ellipsisable text from an indentation tree of a source code program; and associating a high priority value with a first source code line of the indentation tree and a source code line of the indentation tree that is an output value. In one aspect, the large language model comprises a neural transformer model with attention.

[0068] A computer-implemented method is disclosed, comprising: accessing a large language model to perform a task given a hint that fits within the size of a context window; monitoring a source code program in a software development tool; selecting a focus line from the source code program; associating a high priority value with the focus line; associating priority values ​​with non-focus lines based on a distance from the focus line; constructing a hint to the large language model based on the selected line of the source code program; when the size of the hint exceeds the size of the context window, replacing the selected line of the hint with a replacement string based on the corresponding priority value; and applying the hint to the large language model to perform the task.

[0069] In one aspect, selecting the focus line further includes selecting the focus line from a current cursor position in the source code program, from a changed line, from a first line of the source code program, and / or from a line of output data in the source code program.

[0070] In one aspect, when the size of the prompt exceeds the size of the context window, replacing the selected line of the prompt with a replacement string based on the corresponding priority value further includes maintaining indentation associated with the replaced selected line in the prompt. In one aspect, the task is code completion. In one aspect, the large language model is a neural transformer model with attention.

[0071] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0072] It will be understood that the representative methods do not necessarily have to be performed in the order presented or in any particular order unless otherwise indicated. In addition, the various activities described with respect to the methods can be performed in a serial or parallel manner, or in any combination of serial and parallel operations. In one or more aspects, the methods illustrate the operation of the systems and devices disclosed herein.

Claims

1. A system comprising: one or more processors; as well as A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the following actions: accessing a large language model with a fixed-size context window, wherein the large language model generates predictions for the task given a prompt; receiving omissible text, wherein the omissible text comprises an ordered data list, wherein the ordered data list comprises text and source code statement lines having associated priority values; determining a size of the ordered data list; When the size of the ordered data list exceeds the fixed size of the context window: selecting an item in the ordered data list to exclude from the ordered data list based on the priority value associated with each row; and replacing the selected one item from the ordered list of data with a first replacement string; and The hints to the large language model are constructed from the ordered list of the fixed size that satisfies the context window.

2. The system of claim 1 , wherein the one or more programs include further instructions to: submitting the prompt to the large language model; and A response corresponding to the prompt is received from the large language model.

3. The system of claim 1 , wherein the one or more programs include instructions to: The first replacement string is concatenated with an existing replacement string adjacent to the item excluded from the ordered data list.

4. The system of claim 1 , wherein the one or more programs include instructions to: The omissible text is created based on a source code program, wherein an import statement of the source code program and a statement that outputs data from the source code program have a high priority value.

5. The system of claim 1 , wherein the one or more programs include instructions to: The ellipsisable text is created in a source code editor according to a focus line of a source code program, wherein the focus line is associated with a high priority value, wherein the focus line comprises a source code line that has been changed or a source code line at a current cursor position in the source code editor.

6. The system of claim 5, wherein the one or more programs include instructions to: A priority value is associated with the non-focus line based on a distance from the non-focus line of the source code program to the focus line.

7. The system of claim 1, wherein the large language model is a neural transformer model with attention.

8. A computer-implemented method comprising: Access to a large language model that accepts hints within the context window size; Obtaining ellipsisable text comprising text lines and source code statement lines, wherein each of the text lines and each of the source code statement lines comprises a priority value, wherein the large language model is associated with a context window; Determining the sizes of the text lines and the source code statement lines; as well as When the size of the text line and the source code statement line exceeds the size of the context window: selecting one of the text lines and one of the source code statement lines based on a lowest priority value; replacing the selected one of the text lines and the selected one of the source code statement lines with a replacement string in the priority order; and The hints to the large language model are constructed using the data of the omissible text in the priority order.

9. The computer-implemented method of claim 8, further comprising: The omissible text is created according to developer-generated instructions, wherein the developer-generated instructions include a class having a first data element and a method for generating the prompt having the omissible text, wherein the first data element includes the sequence of the text lines and the source code statement lines.

10. The computer-implemented method of claim 8, further comprising: Each replacement string that is adjacent to the replacement term in the omissible text is combined with a single replacement string.

11. The computer-implemented method of claim 10 , further comprising: The indentation of the source code line is maintained within the hint.

12. The computer-implemented method of claim 8, further comprising: creating the ellipsisable text based on a focus line of a source code program, wherein the focus line is a line of the source code that is being altered, wherein the focus line is associated with a high priority value; as well as Priority values ​​are associated with non-focus rows based on their distance from the focus row.

13. The computer-implemented method of claim 8, further comprising: creating the ellipsisable text based on a focus line of a source code program, wherein the focus line is a source code line at a current cursor position in a source code editor, wherein the focus line is associated with a high priority value; as well as Priority values ​​are associated with non-focus rows based on their distance from the focus row.

14. The computer-implemented method of claim 8, further comprising: Creating the ellipsisable text according to the indentation tree of the source code program; as well as A high priority value is associated with the first source code line of the indented tree and the source code line of the indented tree that outputs a value.

15. The computer-implemented method of claim 8, wherein the large language model comprises a neural transformer model with attention.

Images