Length-based large language model

Abstract

The invention provides a large language model based on length. A computer includes a processor and a memory including instructions executable by the processor to receive a prompt for a large language model, the prompt including input text and a target length. The large language model may generate output text based on a length guide vector encoding a target length, the output text including a number of words equal to the target length within user-determined tolerances.

Description

Technical Field

[0001] This disclosure covers training and operating an LLM. An LLM can take in queries in text format (referred to herein as prompts) and output responses in text format. Background Technology

[0002] Computers can include instructions for performing useful data transformations. One class of computer instructions capable of performing various useful data transformations includes Large Language Model (LLM) neural networks. LLMs can be trained on massive natural language datasets to receive natural language queries as input and output responses to those queries.

[0003] An LLM (Local Language Master) is a software program that can be trained on large amounts of data to take natural language queries or prompts as input and output responses. The responses can be in the same natural language as the query, a different natural language, or a structured language such as regular syntax or a computer programming language. A large number (e.g., millions) of samples can be used to train an LLM. In some examples, the responses can closely mimic human natural language responses so that a trained LLM can generally be considered artificial intelligence (AI). In other examples, the output of a trained LLM in response to a natural language query can include structured languages, such as regular syntax or computer languages, that are at or exceed the capabilities of a human user.

[0004] Training an LLM can be accomplished by passing training data through the LLM and comparing the results to the expected results, known as the ground truth. In some examples, the ground truth can be derived from the input data. The LLM encodes the input training data into latent variables and then decodes the latent variables to reconstruct natural language paragraphs based on the parameters. The parameters are updated to minimize the discrepancies by comparing the LLM's output with the input data. In examples, a large amount of training data can be obtained by scanning the internet to obtain natural language samples that include samples of the target natural language. The training dataset can include, for example, news articles, books, social media posts, and technical articles. Summary of the Invention

[0005] Prompts can include requests for responses in virtually any imaginable information domain. Examples of LLM queries include requests for information or history about government agencies, sports teams, or virtual characters. Options included in the input prompt (such as temperature) can control the LLM output to be more random or more deterministic. Another option can include an indication of the size of the text string output by the LLM. While training an LLM to provide output text is relatively easy, getting it to respond to input prompts and output a specific number of words can be more difficult. Training an LLM to output text containing a specific number of words may require additional training with a large number of samples, including real values ​​with the desired number of words. Even with this additional training, the output from the LLM may differ significantly from the number of words requested in the prompt.

[0006] An example of an LLM's use is providing a summary of input text. Providing a summary of input text that reduces the number of words relative to the input text while retaining relevant information from the input text is a useful operation that can be performed by LLM. Generating a summary of input text that includes a specified number of words is a desired result in several applications (e.g., news summaries or directories). Summarizing input text will be used as a non-limiting example of LLM processing in this paper.

[0007] Most LLMs are based on the Transformer architecture. The following is about... Figure 2 This paper describes a Transformer-based LLM. Unless otherwise explicitly stated, references to LLM in this paper refer to Transformer-based LLMs. An LLM can receive both data and commands as input. Inputs to an LLM that can include both data and commands can be referred to as prompts. Prompts are fed into the LLM to generate results. For example, a prompt may cause the LLM to generate a summary of a text document, and may include the text document and commands such as "Summarize the following document in one sentence" or "Summarize the following document in 20 words." Typically, an LLM can receive extensive training with example prompts that include sample input data and input commands to produce usable output. Even with extensive training, results tend to vary considerably, and the LLM may need to be retrained for each newly specified summary length. The technique described in this paper enhances the generation of text summaries through LLM processing by modifying the LLM by adding a length-guided vector. Modifying the LLM by adding a length-guided vector can generate output text summaries that accurately include a specified number of words, while requiring less training, such as fewer computer resources, compared to an unmodified LLM. The length-guided vector can be generated based on the target length included in the input prompt. More accurate and / or more reliable and / or previously impossible LLM outputs can be obtained. See below for more details. Figures 4 to 6The length-guided vector is described.

[0008] This paper discloses a method comprising: receiving a prompt for a large language model, the prompt including input text and a target length; and generating output text in the large language model based on a length-guided vector encoded with the target length, the output text including a number of words equal to the target length within a user-defined tolerance. The input text may include more words than the target length, and the output text is based on the input text. The input text may be received by a tokenizer that generates lexical units representing words in the input text. The large language model may include embedding blocks comprising arrays including lexical vectors and position vectors encoding the positions of the lexical vectors within the array. A decoder may generate a length-guided vector. The length-guided vector encodes the target length by determining a scalar multiple that starts at zero at the origin of the length-guided vector and ends at a one at entries equal to the target length, and by setting the remaining entries in the length-guided vector to zero. The length guide vector can be encoded by determining a scalar multiple of a sine function that starts at zero at the origin of the length guide vector, has a value of one at entries equal to half the target length, and returns to 0 at entries equal to the target length, and setting the remaining entries in the length guide vector to zero.

[0009] The large language model can receive a first target length and a second target length as input, indicating a range of target lengths. The large language model is modified to include a first-length guided vector and a second-length guided vector. The first-length guided vector may include a first sine function that determines a scalar multiple starting at zero at the origin of the first-length guided vector, rising to one at half the first target length, and falling to zero at the first target length, with the remaining entries in the first-length guided vector set to zero. The second-length guided vector may include a second sine function that determines a scalar multiple starting at zero at the origin of the second-length guided vector, rising to one at half the second target length, and falling to zero at the second target length, with the remaining entries in the second-length guided vector set to zero. The first-length guided vector can be added to the second-length guided vector. User-determined tolerances can be selected by the user and determined during the training of the large language model, and can be configured at inference time. User-determined tolerances can be determined by training the large language model. The large language model may be based on a Transformer architecture.

[0010] A computer-readable medium is further disclosed, storing program instructions for performing some or all of the above-described method steps. A computer is further disclosed, programmed to perform some or all of the above-described method steps, the computer including a computer device programmed to: receive a prompt for a large language model, the prompt including input text and a target length; and generate output text in the large language model based on a length-guided vector encoded with the target length, the output text including a number of words equal to the target length within a user-defined tolerance. The input text may include more words than the target length, and the output text is based on the input text. The input text may be received by a tokenizer that generates lexical units representing words in the input text. The large language model may include embedding blocks comprising arrays including lexical vectors and position vectors encoding the positions of the lexical vectors within the array. A decoder may generate a length-guided vector. The length-guided vector encodes the target length by determining a scalar multiple that starts at zero at the origin of the length-guided vector and ends at a 1 at an entry equal to the target length, and by setting the remaining entries in the length-guided vector to zero. The length guide vector can be encoded by determining a scalar multiple of a sine function that starts at zero at the origin of the length guide vector, has a value of one at entries equal to half the target length, and returns to 0 at entries equal to the target length, and setting the remaining entries in the length guide vector to zero.

[0011] The instructions may include other instructions, wherein the large language model can receive a first target length and a second target length indicating a range of target lengths as input, and the large language model is modified to include a first length guide vector and a second length guide vector. The first length guide vector may include a first sine function that determines a scalar multiple starting at zero at the origin of the first length guide vector, rising to one at half the first target length, and falling to zero at the first target length, and setting the remaining entries in the first length guide vector to zero. The second length guide vector may include a second sine function that determines a scalar multiple starting at zero at the origin of the second length guide vector, rising to one at half the second target length, and falling to zero at the second target length, and setting the remaining entries in the second length guide vector to zero. The first length guide vector may be added to the second length guide vector. User-determined tolerances may be selected by the user and determined during the training of the large language model, and may be configured at inference time. User-determined tolerances can be determined by training the large language model. The large language model may be based on a Transformer architecture. Attached Figure Description

[0012] Figure 1 This is a block diagram of an example computer system.

[0013] Figure 2 This is a diagram of an example LLM Transformer architecture layer.

[0014] Figure 3 This is a diagram of an example LLM.

[0015] Figure 4 This is a diagram of an example input embedding vector.

[0016] Figure 5 This is an illustration of an example of a sine length guided embedding.

[0017] Figure 6 This is an illustration of an example of a sine range length guided embedding.

[0018] Figure 7 This is a flowchart of an example process for generating text summaries based on LLM and length-guided embedding. Detailed Implementation

[0019] Figure 1 This is an illustration of an example computing system 100. The computing system 100 includes a computer 110 and a server computer 120 located remotely from the computer 110. One or more computers 110 can receive data about the operation of the computer 110 from a user interface 114. The computer 110 can also operate based on data received from the remote server computer 120. The server computer 120 can communicate with the computer 110 via a network 130 and a network interface 111.

[0020] Computer 110 includes, for example, a processor 112 and a memory 113. Furthermore, memory 113 includes one or more forms of computer-readable medium and stores instructions that can be executed by processor 112 to perform various operations (including those disclosed herein). Computer 110 may include more than one computing device or be communicatively coupled to (i.e., via a communication bus further described below) more than one computing device, such as controllers included in computing system 100 for monitoring and controlling various components. The computer 110 network may additionally or alternatively include known wired or wireless communication mechanisms, such as Ethernet or other communication protocols.

[0021] Additionally, computer 110 can be configured to communicate with remote server computer 120 (i.e., cloud server) via network interface 111 through network 130, as described below. The network interface includes hardware, firmware, and software that allow computer 110 to communicate with remote server computer 120 via network 130 using wired and / or wireless protocols or arrangements. Network interface 111 can therefore include components configured to utilize various wired and wireless networking technologies (i.e., cellular, wireless ... The computer 110 includes processors, memory, transceivers, etc., for Bluetooth Low Energy (BLE), Ultra Wideband (UWB), peer-to-peer communication, UWB-based radar, IEEE 802.11, and other wired and wireless packet networks or technologies. The computer 110 also includes, for example, a known non-volatile medium 116. The computer 110 can record data by storing it in the non-volatile medium 116 for later retrieval and transmission to the server computer 120 via the vehicle communication network and the vehicle-to-network interface 111. The non-volatile medium 116 can also be removable for file storage or transfer to another computer 110.

[0022] As already mentioned, software programs are typically included in instructions stored in memory 113 and executable by processor 112 of computer 110. For example, computer 110 may include programming for operating one or more software programs (such as neural networks). Using data received in computer 110, i.e., input data from user interface 114, server computer 120, etc., computer 110 can make various determinations and output data via display 115 or network interface 111. User interface 114 may include input devices such as keyboards, pointing devices such as mice or touchpads, scanners, cameras, etc. Output data may also be transferred to non-volatile media 116, such as disk drives, flash drives, memory cards, or printers.

[0023] Server computer 120 typically shares features with both the V2I interface 111 of vehicle 110 and computer 110, such as a computer processor and memory and configuration that communicate via network 130, and therefore these features will not be described further to reduce redundancy. Server computer 120 can be used to develop and train machine learning software that can be transferred to computer 110.

[0024] Figure 2 This is an illustration of example LLM 208. Table 1 includes a sample text document 200 that will be tokenized by tokenizer 202 and input into LLM 208:

[0025]

[0026] Table 1. Sample text document 200.

[0027] The text document 200 included in Table 1 can be converted into a list of token vectors 204 by a tokenizer 202, which includes token vectors to be stored in the variable "Input". Example tokenizer 202 commands including parameters of tokenizer 202 are included in Table 2. Tokenizer 202 may include one or more tables that can associate input words with numerical values ​​included in the token vectors; then, LLM 208 can identify the numerical values ​​specified in the token vectors as the corresponding original words. Processing words as token vectors is a more efficient way to process text than storing and moving text strings. Tokenizer 202 takes as input a text string, which may be the text document 200 included in Table 1, and parameters that determine the format of the output token vector list 204, which is labeled "Input" because it becomes the input of LLM 208. After the tokenizer 202 command, a print statement prints the number of tokens included in the "Input" variable on the next line.

[0028]

[0029] Table 2. Tokenizer 202 instructions.

[0030] Following the tokenizer 202, a list 204 of token vectors, including token vectors corresponding to the input text document 200, is input into the LLM 208 along with a target length 206. Each token in the token vector list 204 is a vector. Table 3 shows the loop of inputting the token vector list 204 and the target length 206 into the LLM 208 three times to generate three output summaries 210 for three target lengths 206 containing 75, 100, and 125 words. The token vector list 204 of the text document 200 and the target length 206 are input into the LLM 208. The LLM 208 is labeled in Table 3 as a command named "model.generate" and takes the token vector list 204 of the text document 200 and the target length 206 as the parameter "target_tokens=target" in the second row of Table 3. The LLM 208 produces the output summaries 210 as variables labeled "output" with tokens. The tokenizer 202 executes the `tokenizer.batch_decode` command to convert the tokens in the "output" into words in the variable labeled "results". The tokenizer 202 can also return the number of words in the "output" in the variable labeled "results_len". The print commands in rows 4 through 6 of Table 3 print the output summary, the target length 206, and the actual number of words included in the output summary 210 returned by LLM 208.

[0031]

[0032] Table 3. Inputs for LLM 208

[0033] Table 4 shows the output summaries 210 of the three target lengths 206 in Table 3. The target length 206 is labeled "target lexical," the number of lexical units (words) generated by the LLM 208 is labeled "generated lexical," and the output summaries 210 are labeled "summaries." The number of generated lexical units included in the output text summaries 210 differs from the target length 206 because the LLM 208 has been trained to produce grammatically correct sentences that produce the output summaries 210, which include a number of words within a user-selected percentage added to or subtracted from the target length 206. For example, the first summary 210 is 2.7% less than the target of 75 words, the second summary 210 is 3% more than the target of 100 words, and the third summary is 4.8% greater than the target length 206 of 125 words. The user-determined tolerance can be selected by the user and determined during the training of the large language model, and can be configured at inference time.

[0034]

[0035] Table 4. Output summary 210, target length 206, and actual length.

[0036] Figure 3 This is an illustration of an exemplary Transformer-based LLM 300 neural network. Transformer neural networks can be used to implement LLM 300 to efficiently process large amounts of training data. Training a Transformer neural network may result in storing a large number of parameters for use during inference. The LLM 300 is a neural network optimized for processing sequential data such as natural language. The LLM 300 is an attention-based sequence-to-sequence encoder-decoder architecture that takes natural language text as input, executes instructions included in the input text, and outputs text. Attention-based architectures can handle long-range dependencies, where the outcome can be based on words or phrases that are far apart from each other in the input data. Attention-based architectures also excel at determining context, where context is defined as the potential conditions affecting the outcome from the LLM 300 output. Context can be determined based on more than one part of the sequence input to the LLM 300.

[0037] The Transformer-based LLM 300 neural network is described in the following paper: "AttentionIsAllYouNeed," Asish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, 31 st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA. The technique described in this paper modifies a Transformer-based LLM 300 neural network by adding a target length of 206 that can be applied to the output sequence. The length-guided vector based on the target length of 206 can be repeated for the length of the output sequence and scaled based on the current position.

[0038] In the example of a linear-length guide vector, scale_fn(i) determines the scalar multiple of the vector in the vector array included in the LLM decoder 326 at position i. The decoder vector array included in the LLM decoder 326 determines the scalar multiple of each vector word in the array, since the vectors are repeated such that each word in the sequence has one vector. The remaining elements of the guide vector 412, which has a total length equal to the length of the word array and the length of the position vector 410, are set to zero. An example linear function for determining scale_fn(i) is shown in equation (1):

[0039] scale_fn(i)=(1 / len_guid_emb)*i{i≤len_guid_emb,0 otherwise} (1)

[0040] Where len_guid_emb is the target length, 206. Regarding... Figure 5 and Figure 6 This describes an additional function that can indicate the values ​​of the elements of the length-guided vector 412.

[0041] Very large datasets can be used to efficiently train Transformer-based LLM 300. For example, LLM 300 can efficiently produce results by sampling portions of intermediate results generated by LLM 300. The sampling differences between one run of LLM 300 and the next can lead to LLM 300 being nondeterministic, meaning that LLM 300 can produce different results for more than one run based on the same input cue. While the design of LLM 300 makes strictly deterministic behavior sometimes impossible, careful cue design can produce results close enough to be used for the desired purpose without being exactly the same. Close results are those that indicate the same meaning but are expressed with slightly different wording.

[0042] The technique for generating reproducible results involves performing multiple runs of the LLM 300 for a single cue to produce multiple results. These results can be compared using a voting algorithm, where the most frequently occurring response is selected. The more detailed and specific the examples included in the cue, the more likely it is to produce similar results for repeated applications of the cue. The more detailed and specific the input cue, the less variation the output of the trained LLM 300 will produce. The technique described in this paper for modifying the LLM 300 provides enhancements to the generation of summaries from input text documents by adding a length-guided vector, which reduces the variation in the output summary.

[0043] The LLM 300 is a Transformer-based neural network architecture that includes attention-based processing to efficiently handle large amounts of input text. This attention-based processing encodes the text into numeric tokens in a token vector list 204, which includes an entry for each word in the input text. Embedding block 312 forms a token vector array for the LLM 300 to process by adding positional encodings to the token vector list 204 as input to the LLM 300. The following is about... Figure 4 Describe the embedding block 312 and position encoding.

[0044] LLM 300 processing begins by receiving a token vector array from embedding block 312 in the first decoder block 310 of the LLM 300. The following is about... Figure 4The embedding block 312 is described. The LLM 300 may include multiple encoder blocks 310, followed by multiple decoder blocks 326. The encoder blocks 310 include multiple layers that determine relationships between lexical units extracted from a human-readable text document in a list of lexical vectors 204, based on parameters determined during training. The processed lexical vectors may be output by one or more encoder blocks 310 as latent variables 324, which are called latent variables because they are typically not visible to the user or used outside the LLM 300. The latent variables 324 are then processed by one or more decoder blocks 326, which process the lexical units and relationships included in the latent variables 324 to generate an output lexical summary 346. The output lexical summary 346 may be fed into a tokenizer 302 with appropriate parameters as shown in Table 3 above to generate human-readable text comprising a target length of 306 words, plus or minus a user-determined percentage.

[0045] The position-encoded vector output by embedding block 312 is received by multi-head attention layer 314. Multi-head attention layer 314 is a self-attention processing layer that generates multiple intermediate results in parallel. Multi-head attention layer 314 allows LLM 300 to determine the result based on multiple intermediate results in a single pass. Generating cues that emphasize the relationships between words in the input text, providing context by describing these relationships, and generating multiple intermediate results can enhance the processing efficiency of LLM 300 and reduce the impact of its non-deterministic behavior.

[0046] The self-attention processing of the multi-head attention layer 314 can determine the relationships between words in the input vector without additional external data. Weights or parameters included in the training data of the LLM 300 can determine the relationships between words and the scores of those relationships. The multi-head attention layer 314 can determine multiple relationships and scores between multiple sets of input words in the same process. The multi-head attention layer 314 outputs the relationships and scores to an addition and normalization layer 316, in which each of the relationships and scores is individually added to the position-encoded vector and normalized to keep the values ​​of the intermediate processing results within the LLM 300 within preset limits. This normalization is performed based on the accumulated mean and variance values ​​across the entire training dataset, thereby forcing the results to remain within a defined range of values. Normalization can reduce unwanted results or prevent unwanted results from skewing the training data, which helps training converge to effective results more quickly.

[0047] The corresponding multi-head outputs from the summing and normalization layer 316 are input to multiple feedforward networks 318, 320, where the normalization results are processed in parallel. Feedforward networks 318, 320 are fully connected neural networks that compute linear and / or nonlinear functions based on the input normalized vector data. The outputs from feedforward networks 318, 320 are combined with the outputs from the first summing and normalization layer 316 at the second summing and normalization layer 322. The operation of the second summing and normalization layer 322 is the same as that described above with respect to the first summing and normalization layer 316, where the results are normalized to a selected range to suppress unwanted results and enhance training.

[0048] Encoder block 310 outputs latent vector 324 to subsequent encoders 310 or the first decoder block 326. In the example of the modified LLM 300 as described herein, target length 306 can be input to decoder block 326. The following section discusses... Figures 5 to 6 The target length encoding is described. In the example, the target length can be modified to include a range of possible lengths describing the output lexical summary 346 instead of a single target length 206 with a start and stop length. Upon exiting the final encoder 310, the decoder block 326 receives the latent vector 324 at the first decoder attention layer 328, which flattens the input latent vector 324 to compress the number of connections that make the lexical terms included in the latent vector 324 relevant. After the first decoder attention layer 328, the first decoder summation and normalization layer 330 normalizes the flattened vector data as described above to again allow for fast convergence of results during training. After the first decoder summation and normalization layer 330, the second multi-head encoder-decoder attention layer 332 combines the latent vector 324 with the flattened vector output from the first summation and normalization layer 330. The output from the second multi-head encoder-decoder attention layer 332 is fed into the second decoder summing and normalization layer 334, and then into multiple feedforward networks 336, 338 to process each part of the flattened vector data in parallel. The feedforward networks 336, 338 are fully connected neural networks that compute linear and / or nonlinear functions based on the input flattened vector data. Following the feedforward networks 336, 338, the third decoder summing and normalization layer 340 normalizes the output from the feedforward networks 336, 338, as discussed above.

[0049] The LLM 300 can include multiple decoder blocks 326 (for example, Figure 3A block 326 is shown in the diagram. Following the final decoder block 326, the LLM 300 may include a linear layer 342 that takes the flattened vector data output from the final decoder block 326 as input. The linear layer 342 includes a fully connected neural network that transforms the flattened vector data output from the final decoder block 326 into an output lexical digest 346 for the final output. The LLM 300 can be executed as a software program installed as instructions in the memory 113 of computer 110. The LLM 300 can also be executed as a software program in server computer 120. The input text document 200 can be received from server computer 120 via user interface 114 or via network interface 111. For example, server computer 120 may be included in a networked system such as the Internet.

[0050] The trained LLM 300 can be installed in a computing device where it can be operated to generate useful output text. Operating the trained LLM 300 may include providing prompts to the LLM 300. The prompts are text files containing instructions and data, such as a list 204 of lexical vectors and a target length 206 determining the processing to be performed by the LLM 300. The summary text 346 output from the LLM 300 after processing is based on the content of the prompts received by the LLM 300.

[0051] Figure 4 This is an illustration of example embedding block 312. Embedding block 312 receives a list 204 of token vectors generated by tokenizer 202 from text document 200 as input. Each element 402 in the list 204 of token vectors is a token vector. Embedding block 312 adds the list 204 of token vectors to position vector 404, where each element 406 of position vector 404 is a vector. Embedding block 312 outputs a position-encoded vector 410, where each element is a vector.

[0052] Figure 5 This is a diagram of an example curve 500 of a sine function 502 that can be used to determine the values ​​of the elements of a length-guided vector. The sine function can be used to represent... Figure 3 The described linear function, scale_fn(i), determines the scalar multiple of the entire decoder vector in the list of vectors included in the LLM decoder at position i. In some examples, replacing the linear function with a sine function 502 in the length-guided vector in decoder 326 can enhance the ability of LLM 300 to generate the output text summary 346 of the input text document 200. For example, the sine function 502 can be effective when the target length 206 exceeds 300 words. An example sine function that can replace the linear function in equation (1) is shown in equation (2):

[0053] scale_fn(i)=sin((π / 2len_guid_emb)*i) {i≤len_guid_emb,0 otherwise} (2)

[0054] Where len_guid_emb is the target length, which is 206.

[0055] Graph 500 illustrates the sine function 502 by plotting the number of elements on the x-axis and the element values ​​on the y-axis. The sine function 502 begins at a point where the number of elements 504 is zero and the element value is zero, rises to an element value of one at a point where the number of elements 508 equals half the target length 206, and falls back to an element value of zero at a point where the number of elements 506 equals the target length 206. The length-guided vector, scale_fn(i), determines the scalar multiple of the entire decoder vector included in the list of vectors in the LLM decoder at position i. Figure 6 This is a diagram of example curves 600 for two sine functions 602 and 604. Curve 600 illustrates the two sine functions 602 and 604 by plotting the number of elements on the x-axis and the element values ​​on the y-axis. The two sine functions 602 and 604 are used to indicate the range of the target length 208. In the example using the range of target lengths, the first target length indicates the starting value, and the second target length indicates the ending value. In the input prompt, the target range can be indicated by two target lengths 208 separated by a hyphen. The LLM 300 can then generate an output lexical digest 346, which includes multiple words between the first target length and the second target length.

[0056] The target range can be inserted into the first length guide vector by determining a first sine function 602 for the first target length 206 according to equation (3). The first sine function starts at entry 606, which is equal to the origin with an element value of zero, rises to an element value of one at an entry equal to half the first target length 206, and falls to an element value of zero at entry 608, which is equal to the first target length 206. The remaining elements of the first length guide vector are set to zero. The second target length can be inserted into the second guide vector by determining a second sine function 604 according to equation (3). The second sine function starts at entry 606, which is equal to the origin with an element value of zero, rises to an element value of one at an entry equal to half the second target length 206, and falls to an element value of zero at entry 610, which is equal to the second target length 206. The remaining elements of the second guide vector can be set to zero. The first sine function 602 and the second sine function 604 can be combined by point-by-point addition.

[0057] Figure 7This is a flowchart of a process 700 for generating text summaries of chapters of a document comprising multiple chapters. For example, each chapter of the document may potentially include 5,000 to 20,000 words. Each chapter may be summarized to a target length of 100 to 150 words, where LLM 300 is modified to include input prompts comprising the word-segmented chapters as described herein, such as text document 200 and target length 208. For example, output text summaries 210 may be inserted into a table of contents included at the beginning of the document. For example, process 700 may be implemented in computer 110 or a combination of computer 110 and server computer 120. Process 700 includes multiple boxes that can be executed in the order shown. Alternatively or additionally, process 700 may include fewer boxes and may include boxes executed in a different order.

[0058] Process 700 begins at box 702, where, starting with the first chapter, computer 110 copies the next chapter of the document from the document into input text document 200.

[0059] At box 704, the input text document 200 is fed into the tokenizer 202 to determine the digitized tokens included in the token vector list 204, which represent words included in the text document 200, as mentioned above. Figure 2 and Figure 3 As described in Tables 1 and 2.

[0060] At box 706, the segmented text document 206 and the target length 206 are input into the LLM 300, which is modified to determine a length-guided vector based on the target length 206 and apply the length-guided vector to the output sequence. Then, the LLM 300 generates an output lexical summary 346 based on the input segmented text document 206.

[0061] At box 708, the output lexical summary 346 is input to a tokenizer 202 with appropriate parameters to convert the output lexical summary 346 from lexical units into words. The words output from the tokenizer 202 generate a text summary 210, which includes a target length 206 words plus or minus a user-determined percentage of words.

[0062] At box 710, text summary 210 is output to a software program that formats the output text summary 210 as entries in the table of contents of a document's chapters.

[0063] At box 712, computer 110 checks the document to determine if there are any more chapters to summarize. If there are more chapters, process 700 returns to box 702 to copy the next chapter to input text document 200. If no more chapters remain, process 700 proceeds to box 714.

[0064] At box 714, insert a formatted table of contents from box 710 at the beginning of the research document to provide an output of the table of contents, including an overview of the document's chapters. After box 714, process 700 ends.

[0065] Computing devices such as those described herein typically include commands that can be executed by one or more computing devices such as those identified above and are used to perform blocks or steps of the processes described above. For example, a process block described above can be embodied as a computer-executable command.

[0066] Computer-executable commands can be compiled or interpreted by computer programs created using various programming languages ​​and / or technologies, including but not limited to the following single or combined forms: Java TM C, C++, Python, Julia, SCALA, Visual Basic, JavaScript, Perl, HTML, etc. Typically, a processor (i.e., a microprocessor) receives commands from memory, computer-readable media, etc., and executes these commands, thereby performing one or more procedures, including those described herein. Such commands and other data can be stored in files and transferred using various computer-readable media. Files in a computing device are typically collections of data stored on computer-readable media such as storage media, random access memory, etc.

[0067] Computer-readable media (also known as processor-readable media) include any non-transitory (i.e., tangible) medium that participates in providing data (i.e., instructions) that can be read by a computer (i.e., by the computer's processor). Such media can take many forms, including but not limited to non-volatile and volatile media. Instructions can be transmitted via one or more transmission media, including optical fibers, wires, wireless communications, and internals that constitute a system bus coupled to the computer's processor. Common forms of computer-readable media include, for example, RAM, PROM, EPROM, FLASH-EEPROM, any other memory chip or magnetic tape, or any other medium from which a computer can read.

[0068] Unless otherwise expressly indicated herein, all terms used in the claims are intended to have the ordinary and common meaning as understood by those skilled in the art. Specifically, unless the claim statement expressly limits it to the contrary, the use of singular articles such as “a,” “the,” and “said” should be interpreted as one or more of the elements indicated by the statement.

[0069] The term “exemplary” is used in this document to mean an example, that is, a candidate for “exemplary widget” should be interpreted as referring only to an example of a widget.

[0070] The adverb "approximately" when modifying a value or result implies that the shape, structure, measurement, value, determination, calculation, etc., may deviate from the exact description of the geometry, distance, measurement, value, determination, calculation, etc. due to defects in materials, machining, manufacturing, sensor measurement, calculation, processing time, communication time, etc.

[0071] In the accompanying drawings, the same reference numerals indicate the same elements. Regarding the media, processes, systems, methods, etc., described herein, it should be understood that although the steps or blocks of such processes, etc., are described as occurring according to an ordered sequence, such processes can be practiced by performing the described steps in an order other than that described herein. Furthermore, it should be understood that some steps may be performed simultaneously, other steps may be added, or some steps described herein may be omitted. In other words, the description of processes herein is provided for the purpose of illustrating certain embodiments and should in no way be construed as limiting the claimed invention.

[0072] According to the present invention, a system is provided comprising: a computer including a processor and a memory, the memory including instructions executable by the processor to: receive a prompt for a large language model, the prompt including input text and a target length; and generate output text in the large language model based on a length-guided vector encoded with the target length, the output text including a number of words equal to the target length within a user-defined tolerance.

[0073] According to an embodiment, the input text includes more words than the target length, and the output text is based on the input text.

[0074] According to an embodiment, the instructions include additional instructions for performing the following operation: receiving input text by a word segmenter that generates tokens representing words in the input text.

[0075] According to an embodiment, the large language model includes an embedding block, the embedding block includes an array, the array includes lexical vectors and position vectors that encode the positions of the lexical vectors in the array.

[0076] According to an embodiment, the decoder generates a length-guided vector.

[0077] According to an embodiment, the length guide vector encodes the target length by determining a scalar multiple that starts at zero at the origin of the length guide vector and ends at a single entry equal to the target length, and setting the remaining entries in the length guide vector to zero.

[0078] According to an embodiment, the length guiding vector encodes the target length by determining a scalar multiple of a sine function that starts at zero at the origin of the length guiding vector, has a value of one at entries equal to half the target length, and returns to 0 at entries equal to the target length, and sets the remaining entries in the length guiding vector to zero.

[0079] According to an embodiment, the large language model receives a first target length and a second target length indicating the range of target lengths as input, and the large language model is modified to include a first length guide vector and a second length guide vector.

[0080] According to an embodiment, the first length guide vector includes a first sine function that determines a scalar multiple that starts at zero at the origin of the first length guide vector, rises to one at half the first target length, and falls to zero at the first target length, and sets the remaining entries in the first length guide vector to zero.

[0081] According to an embodiment, the second length guide vector includes a second sine function that determines a scalar multiple that starts at zero at the origin of the second length guide vector, rises to one at half the second target length, and falls to zero at the second target length, and sets the remaining entries in the second length guide vector to zero.

[0082] According to an embodiment, a first-length guide vector is added to a second-length guide vector.

[0083] According to an embodiment, the user-defined tolerance is selected by the user and determined during the training of the large language model, and can be configured at inference time.

[0084] According to the present invention, a method includes: receiving a prompt for a large language model, the prompt including input text and a target length; and generating output text in the large language model based on a length-guided vector encoded with the target length, the output text including a number of words equal to the target length within a user-defined tolerance.

[0085] In one aspect of the invention, the input text comprises more words than the target length, and the output text is based on the input text.

[0086] In one aspect of the invention, the method includes receiving input text by a word segmenter, the word segmenter generating lexical units representing words in the input text.

[0087] In one aspect of the invention, a large language model includes an embedding block comprising an array, the array including lexical vectors and position vectors encoding the positions of the lexical vectors included in the array.

[0088] In one aspect of the invention, the decoder generates a length-guided vector.

[0089] In one aspect of the invention, the length guide vector encodes the target length by determining a scalar multiple that starts at zero at the origin of the length guide vector and ends at an entry equal to the target length, and setting the remaining entries in the length guide vector to zero.

[0090] In one aspect of the invention, the length guiding vector encodes the target length by determining a scalar multiple of a sine function that starts at zero at the origin of the length guiding vector, has a value of one at entries equal to half the target length, and returns to zero at entries equal to the target length, and sets the remaining entries in the length guiding vector to zero.

[0091] In one aspect of the invention, the large language model receives a first target length and a second target length indicating the range of target lengths as input, and the large language model is modified to include a first length guide vector and a second length guide vector.

Claims

1. A method comprising: Receive prompts for a large language model, the prompts including the input text and the target length; as well as Based on a length-guided vector that encodes the target length, output text is generated in the large language model. The output text includes the number of words equal to the target length within a user-defined tolerance.

2. The method of claim 1, wherein the input text comprises more words than the target length, and the output text is based on the input text.

3. The method of claim 1, further comprising receiving the input text by a word segmenter, the word segmenter generating lexical units representing words in the input text.

4. The method of claim 1, wherein the large language model includes an embedding block, the embedding block includes an array, the array includes lexical vectors and position vectors that encode the positions of the lexical vectors in the array.

5. The method of claim 1, wherein the decoder generates the length guiding vector.

6. The method of claim 1, wherein the length guiding vector encodes the target length by determining a scalar multiple that starts at zero at the origin of the length guiding vector and ends at an entry equal to the target length, and by setting the remaining entries in the length guiding vector to zero.

7. The method of claim 1, wherein the length guiding vector encodes the target length by determining a scalar multiple of a sine function that starts at zero at the origin of the length guiding vector, has a value of one at entries equal to half the target length, and returns to 0 at the entries equal to the target length, and sets the remaining entries in the length guiding vector to zero.

8. The method of claim 1, wherein the large language model receives a first target length and a second target length indicating the range of the target length as input, and the large language model is modified to include a first length guide vector and a second length guide vector.

9. The method of claim 8, wherein the first length guide vector comprises a first sine function that determines a scalar multiple that starts at zero at the origin of the first length guide vector, rises to one at half the first target length, and falls to zero at the first target length, and sets the remaining entries in the first length guide vector to zero.

10. The method of claim 9, wherein the second length guide vector comprises a second sine function that determines a scalar multiple that starts at zero at the origin of the second length guide vector, rises to one at half the second target length, and falls to zero at the second target length, and sets the remaining entries in the second length guide vector to zero.

11. The method of claim 10, wherein the first length guide vector is added to the second length guide vector.

12. The method of claim 1, wherein the user-determined tolerance is selected by the user and determined during training of the large language model, and can be configured at inference time.

13. The method of claim 1, wherein the user-determined tolerance is determined by training the large language model.

14. The method of claim 1, wherein the large language model is based on the Transformer architecture.

15. A method comprising a computer programmed to perform the method as claimed in any one of claims 1 to 14.

Images