Zero-shot strategies for length-controllable summarization

Abstract

Methods and systems control length of generated content summaries through a zero-shot approach. The methods and system are configured to generate modified instructions for existing LLM models but cannot train the existing LLM model because they do not have access to the model internals.

Description

PATENT Docket No. 242616PCT IN THE UNITED STATES RECEIVING OFFICE PATENT COOPERATION TREATY (PCT) APPLICATION FORZERO-SHOT STRATEGIES FOR LENGTH-CONTROLLABLE SUMMARIZATIONInventors: Fabian Retkowski and Alexander WaibelPRIORITY CLAIM

[0001] The present application claims priority to United States provisional patent application Serial No. 63 / 739,830, filed December 30, 2024, titled “Zero-Shot Strategies for Length-Controllable Summarization.”BACKGROUND

[0002] Text summarization has seen remarkable advancements with large language models (LLMs), which can now generate coherent, high-quality summaries that often rival human-written summaries, particularly demonstrated in the extensively studied news domain (Pu et al., 2023; Retkowski, 2023; Zhang et al., 2024; See List of Full Citations provided at paragraph

[0132] ). Consequently, general abstractive summarization tasks are increasingly viewed as ‘solved,’ pushing the research frontier toward more specialized challenges such as controllable summarization, where summaries must adhere to constraints like length, style, or content focus.

[0003] Length control, in particular, represents a crucial capability for practical natural language processing (NLP) systems, as real-world applications frequently demand outputs that meet strict length constraints-from social media posts to UI text, subtitles, and copywriting (Hori et al., 2002; Waibel et al., 2005; Waibel and Fuegen, 2012; Liang et al., 2024; Liu et al., 2024; Zechner et al. 1998). However, while controllable generation performs well in flexible, user-driven tasks involving keywords or specific aspects (Goyal et al., 2022; Xiao et al., 2023; Yang et al., 2023), adherence to strict numerical or length constraints remains problematic (Sun et al., 2023). Studies indicate that even advanced models like Claude 3 Opus and GPT-4 struggle with rigid length control, particularly in zero-shot settings (Chen et al., 2024b; Yuan et al., 2024).

[0004] Traditionally, length-controllable generation has been achieved through decoding adjustments and learning-based methods, often requiring additional training data or model modifications (Hori et al., 2002; Kikuchi et al., 2016; Makino et al., 2019; Yu et al., 2021; Liu et al., 2022). However, they frequently either lead to degradation in quality or are not1604215198.1Docket No. 242616PCTreadily applicable to pre-trained language models.

[0005] Previous research on length controllability has generally been narrow, focusing on a single control scenario defined by a single length measure, such as word count (Juseon-Do et al., 2024; Yuan et al., 2024). Moreover, most approaches rely on fine-tuning models with additional training data or modifying model architectures, such as length-aware attention mechanisms. This shift towards fine-tuning uses additional data on downstream tasks, including supervised fine-tuning, reinforcement learning from human feedback (RLHF) or direct preference optimization (DPO), and parameter-efficient fine-tuning (PEFT) methods such as low-rank adaptation (LoRA) (Jie et al., 2024; Juseon-Do et al., 2024; Yuan et al., 2024). They often require substantial computational resources, access to model internals, and sacrifice the generalizability of the model. Although these methods can be effective, their dependence on model internals and substantial computational resources limits their practicality in real-world settings where only inference APIs are accessible.

[0006] This is a critical gap in zero-shot training in which machine learning can be utilized to perform length controllable without prior ground truth examples. Thus, there is a need for a system that employs a zero-shot training method that uses existing LLMs for length-controllable summary generation and does not rely on access to model internals for fine tuning.SUMMARY OF THE INVENTION

[0007] In a general aspect, the present invention is directed to systems and methods for generating text summaries using summarization models such as large language models (LLMs), speech language model, or vision language model, with precise control over summary length in zero-shot settings — that is, without requiring model fine-tuning, additional training data, or access to internal model parameters. In particular, the present invention provides a framework for length-controlled summarization across diverse length parameters, such as number of words, characters, tokens, sentences, or bullet points, by employing a configurable inference-time architecture compatible with publicly available summarization models accessed via APIs.

[0008] In one aspect, a method according to the present invention comprises receiving an input text and a length constraint defined by a length parameter and a target length value; generating one or more candidate summaries using one or more summarization models; evaluating each candidate summary for compliance with the target length based on the21604215198.1Docket No. 242616PCTspecified parameter; and selecting, revising, or re-generating summaries to produce a final summary that satisfies the length constraint.

[0009] In various embodiments, the system may include one or more of the following modules: a length approximation module that translates the target length from a less controllable measure (e.g., character count) to a more controllable proxy measure (e.g., word count) using statistical mappings; a target adjustment module that compensates for systematic model biases by adjusting the effective target length based on a learned regression function; a sample filtering module that selects the candidate summary with the minimal deviation from the target length among multiple generated outputs; and an automated revisions module that iteratively prompts the summarization model to revise a non-compliant summary until it meets the length constraint within a defined tolerance.

[0010] In some implementations, the inventive system executes integrated strategies that combine two or more of these modules in a structured pipeline. For example, the system may apply length approximation followed by sample filtering (LA-SF), or combine length approximation, target adjustment, and automated revision (LA-TA-AR), depending on the desired balance among summary quality, computational efficiency, and adherence precision.

[0011] In certain variations, the system may support qualitative length targets such as "short," "concise," or "verbose," which are interpreted as approximate quantitative targets based on empirical mappings. Additionally, the system may employ predefined prompt templates to improve instruction-following by the LLM, and may apply custom lengthmeasurement functions tailored to each parameter.

[0012] Embodiments of the present invention address several technical problems in the field of text generation using summarization models. First, they enable enforcement of strict length constraints — particularly for granular measures like characters or tokens — using only zero-shot prompting. Second, they overcome the limitations of conventional approaches that rely on fine-tuning or model internals, which are often computationally expensive or unavailable. Third, they mitigate inherent LLM biases toward under- or over-generation in response to fixed-length prompts. By providing a modular, API-compatible architecture for length-controlled summarization, the invention facilitates practical deployment in real-world applications including user interface copy, subtitles, document abstractions, and contentlimited displays. Additional advantages and features will become apparent from the detailed description that follows.31604215198.1Docket No. 242616PCTBRIEF DESCRIPTION OF THE DRAWINGS AND APPENDICES

[0013] Various embodiments of the present invention are described herein by way of example in connection with the following figures and appendices.

[0014] Figure 1 shows a block diagram for a summary generation system configured to control the length of generated content for a large language model, according to various embodiments of the present invention.

[0015] Figure 2 shows a logic flow diagram for a length controlling method, according to various embodiments of the present invention.

[0016] Figure 3 shows a table for a set of length constraints comprising length parameters and corresponding length targets, that may be provided as input parameters, according to various embodiments of the present invention.

[0017] Figure 4 shows a table summarizing qualitative length qualifiers that may be used as target length values for the length control parameters, according to various embodiments of the present invention.

[0018] Figure 5 shows a table comparing results of length controllability across different length control parameters, according to various embodiments of the present invention.

[0019] Figures 6A-6C show example prompt templates, according to various embodiments of the present invention.

[0020] Figure 7 shows a plurality of count functions that may be configured to measure the length of a generated summary, according to various embodiments of the present invention.

[0021] Figure 8 shows a table summarizing perplexity score for length control parameters including characters, tokens, and words, according to various embodiments of the present invention.

[0022] Figures 9A-B show tables comparing the influence of response priming and sampling temperature on the length adherence of length control parameters, according to various embodiments of the present invention.

[0023] Figure 10A shows a scatter plot of a plurality of data points representing generated summaries, according to various embodiments of the present invention.

[0024] Figure 10B shows a scatter plot of a plurality of data points representing generated summaries based on input words length (x-axis) in comparison to the length deviation of the generated summaries, according to various embodiments of the present invention.

[0025] Figure 11 shows a scatter plot of a plurality of data points representing generated summaries, according to various embodiments of the present invention.41604215198.1Docket No. 242616PCT

[0026] Figures 12A-B show tables summarizing the performance evaluations for each length control approach, according to various embodiments of the present invention.

[0027] Figures 13A-B show tables of performance evaluation data for length approximation and target approximation based on length compliance and length deviation in comparison to baseline performance evaluation data, for the YTSEG dataset and the CNN / DM dataset, according to various embodiments of the present invention.

[0028] Figures 14A-B show line graph plots comparing length compliance rates versus the number of samples and revisions for a plurality of target length parameters, according to various embodiments of the present invention.

[0029] Figures 15A-C show evaluation results for the sample filter system as a function of sample counts on the character length, token length, and word length parameters, according to various embodiments of the present invention.

[0030] Figures 16A-C show evaluation results for iterative automated revisions by the automated revisions module as a function of sample counts on the character length, token length, and word length parameters, according to various embodiments of the present invention.

[0031] Figures 17A-C show evaluation results for sampled revisions as an integrated method as a function of sample counts on the character length, token length, and word length parameters, according to various embodiments of the present invention.

[0032] Figure 18 shows an average output of a plurality of summaries obtained using the GPT-4o model, according to various embodiments of the present invention.

[0033] Figures 19A-C show length approximation and target adjustment evaluations for the plurality of length parameters including character length, token length, word length, according to various embodiments of the present invention.

[0034] Figures 20A-D show evaluations of target length deviation by comparing the word length deviation of generated summaries against input target lengths for a plurality of LLM models, according to various embodiments of the present invention.

[0035] Figures 20E-H show evaluations of target length deviation by comparing the word length deviation of generated summaries against input word length of the original content for a plurality of LLM models, according to various embodiments of the present invention.

[0036] Figures 21 A-C show baseline performance comparisons across different models, according to various embodiments of the present inventions.51604215198.1Docket No. 242616PCTDETAILED DESCRIPTION

[0037] The present invention describes methods and systems to generate summaries with existing large language models (LLM) based on multiple length control parameters (e.g., measures) including words, characters, tokens, sentences, and bullet points, in contrast to the prior studies that focus narrowly on controlling sentences or words. The methods and systems described herein do not rely on ground truth training data and provide a zero-shot approach to controlling length of generated summaries in existing LLMs. Additionally, the methods and systems cannot retrain the existing model because they do not have access to the internals of the existing LLMs.

[0038] Typically, LLMs struggle with precise length control, particularly in zero-shot settings. The methods and systems provide a configurable approach for evaluating LLMs’ length control capabilities across multiple parameters to improve controllability. Experiments conducted by the inventors with LLaMA 3 reveal stark differences in length adherence across measures and highlight inherent biases of the model. To address these challenges, we introduce a plurality of length control approaches including length approximation, target adjustment, sample filtering, and automated revisions. By combining these approaches, the systems and methods substantial improvement in length compliance while maintaining or enhancing summary quality, providing highly effective zero-shot strategies for precise length control without the need for model fine-tuning or architectural changes. The evaluations of the plurality of approaches provide a deeper understanding of LLM behavior in controlled text generation but also pave the way for more reliable and adaptable summarization systems in real-world applications.

[0039] Figure 1 shows a block diagram for a summary generation system 100 configured to control the length of generated content produced by summarization model (e.g., large language models (LLMs), vision-language models (VLMs), speech-language models (SLMs)), according to various embodiments of the present invention. The summary generation system 100 comprises a computer system 102 configured to communicate with a model server 104 hosted by an external computing system (e.g., LLM server system). The computer system 102 may be a web server, client device, or intermediary device that is configured to receive and process content summary requests. The content summary requests may be discrete requests that are received through a user interface with a static or fixed pregenerated content or may be based on real-time content (e.g., captures or recordings of realtime audio or video). The content summary requests may comprise input constraints such as length control parameters and target length values, in addition to source content (e.g., text,61604215198.1Docket No. 242616PCTspeech / audio, images, or video) or an identifier of source content for which the model server accesses the source content to generate an initial summary. The content summary request may comprise a selection of a specific summarization model, or may be left to the computer system 102 to select the best model for the content and input constraints in the request. Upon reaching a threshold compliance or accuracy with the length control parameters, the computer system 102 may be configured to output, display, or return a length-controlled summary in response to the content summary request.

[0040] The summary generation system 100 is configured to perform a length controlling method with various summarization models, such as general purpose summarization modes, LLMs (e.g., LLaMA3, GPT models from OpenAI), other open source models, and / or specialized LLM (e.g., Gemini from Google DeepMind). The summarization model may generate sample summaries based on either abstractive summarization or compression-based shortening. For abstractive summarization, the summarization model generates a summary by rephrasing and interpreting the source content using “natural language processing or natural language understanding.” This approach is traditionally used by LLMs, VLMs, and SLMs when generating human-like summaries. In contrast, compression-based shortening removes certain aspects of the original content that are deemed less important but retains a key concept or phrase in order to preserve the essential meaning of the source content.Compression-based shortening is helpful to convey a main concept or idea without extraneous words used in natural language. The computer system by be configured to select a specific summarization model based on the source content.

[0041] The computer system 102 comprises one or more processors and a memory unit, where the one or more processors execute a plurality of length control modules for generating content based on length control parameters. The plurality of length control modules include a length approximation module 108, a target adjustment module 110, a sample filtering module 114, and an automated revisions module 116. The plurality of length control modules may be implemented with a programmed computer system (e.g., computer system 102), which may comprise one or more computer devices in communication with the summarization model 104, hosted by an external computer system 105 (e.g., model server).

[0042] The one or more computer devices, external computer system, and client device may be implemented through servers, laptops, and / or mobile computing devices. The computer system 102 may comprise multiple computer devices, where the computer devices may be connected and communicate via an electronic data network, such the Internet, a LAN, a WAN, a mesh network, etc. The computer system 102 may communicate with the71604215198.1Docket No. 242616PCTsummarization model 104, hosted by an external computer system 105, via an application program interface (API). The communications between the computer system 102 and the model server 105 may comprise JSON data. In various aspects, the model server 105 may use advanced neural network architectures, typically based on transformers, to process and generate text.

[0043] Figure 2 shows a logic flow diagram for a length controlling method 200, according to various embodiments of the present invention. The summary generation system 100 may initiate a method, at step 202, by receiving input text and tokenizing the input length constraints. The input length constraints may comprise a plurality of length parameters (e.g., 100 chars, 65 tokens, 50%, 2 sentences, 80 words, 5 bullets) (see Figure 3). One of the input length constraints may identify the number of tokens used to tokenize the input text, which breaks down the text into smaller units such as words, subwords, or characters. These tokens are transformed into numerical representations through an embedding layer, which encodes their meaning in a way the model can understand. The model’s architecture may include transformers, a type of neural network designed for handling sequential data. Within the transformer, a self-attention mechanism may be used to identify relationships between words in the input, allowing the model to focus on the most relevant parts of the text when it is generating responses. Positional encoding may also be added to the token representations to help the model understand the order and structure of the sequence. The tokens are passed through multiple layers (e.g., approximation module 108, target adjustment module 110, sample filtering module 114, automated revisions module 116) of the transformer, where each layer refines their representations by capturing more complex and abstract patterns. The summary generation system 100 may determine a specific method as shown by the alternative arrow paths within the plurality of length control modules. For example, the summary generation system 100 may define six different sub-methods (e.g., SR, SF-AR, LA-SF, LA-AR, LA-SF-AR, LA-TA-SF) of sample processing based on the source content (e.g., text, audio, video), source content format (e.g., real-time content / dynamic, static sample / pre-recorded), and length constraint parameters (values for chars, tokens, percentage of reduction or remaining, sentences, words, bullets). The computer system may be configured to automatically determine the source content format or may be a pre-configured parameter.

[0044] The input length constraints may undergo preprocessing, at 204 and 206, though the length approximation module 108 and / or target adjustment module 110. The computer system 100 can generate new request information based on the pre-processing, and at 208, may then generate modified instruction for the summarization model 104. The modified 81604215198.1Docket No. 242616PCTinstructions are transmitted to the model summarization server 105. The computer system, at 210, receives a generated sample and evaluates the results to determine whether it meets a threshold or range for summarized content. If the generated content is within a threshold or range for length control metrics or the length control parameters, the content summary is provided, at 216, as the output.

[0045] The computer system may employ a model that performs a plurality of sample iterations before a final sample is generated. Before various iterations of samples, the computer system may generate new modified instructions before transmitting a prior sample for revisions to the summarization model. Additionally, the computer system 102 may perform, at 212 and 214, post-processing of a generated sample through the sample filtering model 114 and the automated revisions module 116.

[0046] This process enables the summarization model 104 to grasp nuances in language, context, and meaning. At the output stage, the model predicts the next token in the sequence by evaluating probabilities for all possible tokens and selecting the most likely one. This prediction process continues iteratively, generating text until a stopping condition is met. The final tokens are then decoded back into human-readable text, forming the output response. By leveraging billions of parameters and extensive training on diverse datasets, summarization models (e.g., large language model, speech language model, vision language model) are capable of understanding and generating coherent, contextually appropriate text across a wide range of tasks. In various embodiments, the summarization model 104 may be hosted locally within the computer system, in which case the applications of the computer system can interact directly with the summarization model through libraries or frameworks. The input text may be passed as tokenized text or numerical tensors.

[0047] The present invention further describes methods and systems for zero-shot length controllability in for summarization models across multiple length measures (e.g., parameters). The computer system 102 may be configured to implement various methods to enable LLMs to generate summaries with precise lengths. The computer system 102 may evaluate the inherent length control capabilities of summarization models across a multitude of parameters and length targets, identify significant disparities in performance for controlling length, and implement instructions to modify the model behaviors and biases that may contribute to generated summary lengths.

[0048] The computer system 102 may control length through various sub systems including length approximation 108, target adjustment 110, sample filtering 114, and automated revisions 116, to enhance length adherence in zero-shot settings while preserving 91604215198.1Docket No. 242616PCTor enhancing summary quality. The summary generation system 100 may employ various combinations of the different length control modules yielding highly effective ‘recipes’ for length-controllable summarization with content summarization models.

[0049] The summarization models 104 exhibit varying degrees of proficiency in controlling different length measures, potentially excelling in one (e.g., words) while struggling with others (e.g., characters). To address this disparity, the length approximation module 108 may be configured to translate length constraints between different length control parameters (e.g., sentence length, word length, token length, character length, bullet point count), leveraging the model’s proficiency with one parameter measure to improve its performance of another parameter.

[0050] The target adjustment module 110 may generate instructions that configure the summarization model 104 to generate a summary based on a target length set by the input length constraints. However, conventional models have noted the challenges of setting target values and length bias in summarization models. In one example, an summarization model was configured to generate summaries with a length target of 50 words, but the summarization model consistently generated outputs that were longer than the target values. In order to compensate for exceeding the target length, the summarization model was reconfigured for a target length of 25 words, to meet a 50-word requirement. Although anecdotal and not systematically analyzed, this illustrates the length deviation addressed by the target adjustment module 110.

[0051] The sample filtering module 114 draws inspiration from similar systems that are used in code generation models, where code samples are filtered based on passing unit tests. More broadly, this methodology represents a form of re-ranking that has been effectively applied in various domains. For instance, the sample filtering module 114 may employ machine translations and dubbing, which can be used to rank translations based on their alignment with source audio duration.

[0052] The automated revisions module 116 may perform a self-correction scheme to refine the generate summaries by refining generated text with provided feedback, potentially in multiple passes. In domain-specific tasks like machine translation, post-editing with the LLM models 104 has shown significant promise. Thus, the automated revisions module 116 aligns with this line of research by enabling the LLM model 104 to adjust its outputs to better meet length constraints provided by the input length constraints.

[0053] The method of Figure 2 may summarize content for various requests including existing content (e.g., document or prerecorded media) summary requests through a prompt 101604215198.1Docket No. 242616PCTor user interface, real-time language translation (e.g., multi-linguality, cross-linguality) requests, or real-time shortening requests for transcribed content (e.g., subtitle or captions for dynamic media). For example, the computer system 102 may be configured, as part of a video streaming or broadcasting platform, to provide a plurality of summary outputs in realtime as visual text such as captions or subtitles, concurrently with an associated audio or video stream. The associated audio or video stream may be treated as the source content for shortened text outputs. The computer system may generate one or more content summarization requests as sentence-wise summaries where the generated output, such as shortened captions or subtitles, are representative of the source content but shorter in length. Unlike traditional subtitles, captions, or transcripts that provide real-time verbatim text of concurrent audio, the generated content may be sentence fragments that are faster to read and only convey a main phrase from the source. This form of summarization is helpful for realtime content where the consuming-audience places a higher importance on the speed of comprehension rather than a human-like format.

[0054] In another example, the computer system 102 may be configured as part of an audio, video, or conferencing platform (e.g., video conference, VOIP) to receive a content summarization request as real-time audio or video streams in a first language and provide a real-time summary in second language. The computer system 102 may generate real-time subtitles in a second language through either output-language compression or input-language compression. For output-language compression, the computer system 102 accesses or produces source subtitles of in the first language and generates subtitles in the second language with a natural language translation. Once the subtitles are in the second language, the computer system can employ a summarization model to shorten the subtitles based on length control parameters. For input-language compression, the source subtitles are first shortened in the first language, according to the length control parameters. Once the computer system generates shortened subtitles, it then translates the subtitles into the second language. Based on the implementation by the computer system, the language translation aspect may be performed as pre-processing or post-processing for a generated sample.

[0055] In another example, the length control system can be used for cross-language text-to-speech or speech-to-speech use cases. In this configuration, when source text is translated into a target language and synthesized into speech, the computer system monitors the speech duration of the output, based on a predetermined speech cadence. The computer system may set a threshold duration at the speech cadence for the output speech. If the output speech exceeds the target duration threshold, rather than increasing the speech cadence, the computer 111604215198.1Docket No. 242616PCTsystem incrementally shortens the source text in its original language according to the length control parameters. The computer system may then retranslate the shortened input, and regenerates the speech output. This iterative process ensures that the final spoken summary in the target language satisfies both length constraints and temporal duration constraints. In the speech-to-speech context, the system would operate in the same manner except that the original text in the source language is first generated based on input speech.

[0056] Figure 3 shows a table for a set of length constraints 300 comprising length parameters 302 and corresponding length targets 304, that may be provided as input parameters (as shown in Figures 1 and 2), according to various embodiments of the present invention. The set of length constraints 300 comprise a plurality of length parameters 302 and corresponding length targets 304. For example, the set of length constraints may comprise length control parameters (e.g., measures) including the number of words, number of characters, number of tokens, number of sentences, and / or number of bullet points. Each of the length parameters may be associated with a plurality of corresponding length targets that can be used to control the length of a generated summary. The set of length targets enables a multifaceted assessment of length control across varying granularities.

[0057] In various aspects, the length parameters may be used for specifying and measuring summary length in controllable text summarization. The length parameters may be categorized into two fundamental groups including structural parameters and granular parameters. The structural parameters define summary length using higher-level textual structures, such as sentences, bullet points, or paragraphs. The granular parameters define summary length using fine-grained linguistic or technical units such as characters, tokens, or words.

[0058] Figure 4 shows a table 400 summarizing qualitative length qualifiers that may be used as target length values for the length control parameters, according to various embodiments of the present invention. The computer system 102 may be configured to receive qualitative length qualifiers rather than integer values for target values or specific length control parameters. As shown, the qualitative length qualifiers are compared based on compliance rare, number for sentences generated, number of words generated, number of tokens generation, number of characters generated and sentence length. Figure 4 further shows the potential the impact of qualitative length quantifiers on such as “short”, “concise”, “brief’, “moderate”, “medium-length”, “comprehensive”, “verbose”, and “long” on summary length control parameters. Further analysis of the qualitative length qualifiers is described with respect to Figure 11.121604215198.1Docket No. 242616PCT

[0059] Figure 5 shows a table 500 comparing results of length controllability across different length control parameters, according to various embodiments of the present invention. The length controllability metrics are calculated for a plurality of target length values for each length control parameter. The various metrics may be used to determine the accuracy of each generated summary, in each row, corresponding to a length target value.

[0060] The length approximation module 108 may comprise a plurality of length approximation modules including character length and token length approximation modules. The character length approximation module may be configured to approximate characterlevel control through word-level targets that employs a statistical mapping between character and word counts derived from generated summaries. The character length approximation module may useto denote the mean word length in characters. The sample summaries, shown in Figure 5, are generated based on the mean word length in characters, as determined by -a> = 6.31 ± 0.50. The character-to-word conversion is defined as factor ac^ _, as:The character length parameter may then be approximated by substituting the original target character count Ctarget with a target word count JFtarget:

[0061] The token length approximation module may be configured in a similar manner as the character length approximation module, but for token-level control. The token length approximation module determines a mean word length in tokens as / zt= 0.798 ± 0.0656 tokens per word. The token-to-word conversion factor is thus defined as:Analogous to the character length approximation, the computer system 102 calculates the target word count from a given token count:

[0062] The summarization models often systematically deviate from specified target lengths within the same length parameters due to inherent biases, leading to systematic overestimation or underestimation. The computer system 102 may employ the target adjustment module 110 to correct this overestimation or underestimation by refining the initial target length based on observed deviations. For example, a word target adjustment module uses the word length parameters based on quantifiable relationships between target length and length deviation. The word target adjustment module derives a third-degree polynomial model, based on nonlinear patterns, that quantifies the relationship between target 131604215198.1Docket No. 242616PCTlength and length deviation. The third-degree polynomial model may be represented by:W target=tZ + / ’JFtarget + cW target + dW^ target (5)

[0063] The computer system calculates the length approximation coefficients in Equation 5, by aggregating the word counts and the character counts across the generated summaries from the LLaMA3 baseline experiments on YTSEG (see Figure 5). The computer system 102 may then compute the average word length in characters and the average number of tokens per work across these summaries.

[0064] The target adjustment module 110 derives a polynomial regression based on generated summaries of the YTSEG dataset with random word targets between 25 and 300 words. For word targets, the empirically determined coefficients are:W target=23.7904 + 4.3 X1O’5^target + 1.226 X IO’2w2target - 3.3 X 10’5W3target (6) where IF target is the adjusted word target and J target is the original word target.

[0065] The sample filtering module 114 may be configured to enhance the precision of length control. The sample filtering module 114 leverages the natural variability in LLM outputs by filtering from multiple generated candidates based on the search space of possible summaries and selecting the one that best complies with the specified length constraints. The sample filtering module 114 allows the computer system 102 to improve adherence to target lengths without compromising semantic content. For example, the LLM model 104 generates a set of N candidate summaries Sj, &, ... , 5N , for each input text, by sampling from the LLM model 104. The sample filtering module 114 calculates the length / .fS';) of each summary in the specified parameter and computes its absolute deviationfrom the target length Ztarget:i=I L(Si) — / .target | (7)The summary that minimizes this deviation is selected as the best summary S* :„* > argminSjSiThe sample filtering module 114 enhances length compliance while maintaining content quality by selecting the summary with minimal length deviation.

[0066] The automated revisions module 116 avoids typical problems experienced in refinement schemes such as issues with the quality of the feedback in the LLM. In various aspects, the automated revisions module 116 may be configured to ensure the generated summaries meet the requirements of the input constraints by refining non-compliant summaries generated by the LLM model 104. For example, the computer system 102 may leverage automatic length evaluation to provide guidance to the summarization model 104,141604215198.1Docket No. 242616PCTand steer it towards more length-compliant summaries. If an initial summary So violates length constraints, the system may be configured to prompt the summarization model to revise the non-compliant summary and generate a revised summary 5i. The system can accurately determine length compliance which mitigates the issues associated with feedback quality and instead provides precise feedback.

[0067] The computer system 102 may evaluate the compliance of a generated summary based on a compliance threshold, e, as a percentage of the target length. The computer system 102 may determine that a summary S is non-compliant if its length L(S) deviates from the target length / .target by more than .| L(S) — / .target | > f L target (9)This determination that the summary S is non-compliant may trigger the automated revision process.

[0068] The automated revisions module 116 may operate iteratively if the initial revised summary fails to meet the length constraint, set by the length measures. In each iteration z, the LLM 104 generates a new summary Si based on the previous Si-i, continuing until the summary complies with the length constraint or a maximum number of revisions N is reached. To prevent context length from exceeding the LLM’s capacity, each revision step is treated independently, using the latest summary as input and applying a consistent prompt template.S = LLM( 7; 5i-l, Ztarget) (10)where Si is the zthrevision, T is the original text, and St- is the previous summary.

[0069] The computer system 102 may generate content summaries through different integrated methods that include different combinations of the length control modules to generate a summary that meets the input length parameters. As shown in Figure 1, the plurality of length control modules comprise the length approximation module 108, target adjustment module 110, sample filtering module 114, automated revisions module 116, and sampled revisions module 118 (e.g., integration of sample the filtering module 114 and the automated revisions module 116). The sampled revisions module 118 combines the sample filtering module 114 and the automated revisions module 116 like the SF-AR method.However, the SF-AR method performs a sequential or series processing operation whereas the sampled revisions module 118 performs a plurality of iterative samples that are revised through the the sample filtering module 114 and the automated revisions module 116. The plurality of length control modules may be implemented at a series to instructions stored in non-transitory memory or hardware circuits within the computer system 102.151604215198.1Docket No. 242616PCT

[0070] The computer system 102 may be configured to execute a first method (LA-SF) incorporating the length approximation module 108 and sample filtering module 114. For a target length / .target in measure Mi (i.e., a first modified length control parameter), the length approximation module 108 first approximates the target length in measure Mi , modifies the summary generation instructions, or content summary instructions, for the summarization model 104, and the sample filtering module 114 receives the generated N candidate summaries {Si, ... , SN} from the summarization model 104. The sample filtering module 114 continues to select the best summary S* based on the original target length in Mi. As used herein, “content summary instructions” may refer to a structured set of one or more prompts, directives, or input strings generated based on the source content and length control parameters, and designed to guide the summarization model in generating the content summary. These may optionally be based on predefined prompt templates, adjusted length constraints, or revision strategies

[0071] The computer system 102 may be configured to execute a second method (LATA) incorporating the length approximation module 108 and target adjustment module 110. This method processes the length control parameters with the length approximation module 108 and then passes the processed data (e.g., modified length control parameters) to the target adjustment module 110. Given a target length / .target in measure Mi (i.e., a first length control parameter), the computer system 102 first approximate the target length / .target in measure Mi (i.e., a second length control parameter), then adjust the approximating target L ’target using the polynomial regression model as the final substituting target L ’’target. The computer system 102 then generates modified instructions for the summarization model to generate a summary. The regression functions used in the length approximation and target adjustment modules may be derived empirically from training data comprising prior summarization model outputs and their observed length properties. In some implementations, these may be fit using leastsquares or regularized regression techniques, and may be periodically retrained to reflect evolving summarization model behavior.

[0072] The computer system 102 may be configured to execute a third (LA-TA-SF) method incorporating length approximation module 108, target adjustment module 110, and sample filtering module 114. This method combines all three systems sequentially. The computer system 102 first applies the length approximation module 108 and target adjustment module 110 as described in the second method, and then generates a set of modified instructions for the summarization model 104. The summarization model 104 generates N candidate summaries (i.e., samples) based on the adjusted target length / .target in 161604215198.1Docket No. 242616PCTthe modified instructions. The sample filtering module 114 receives the N candidate summaries and filters the N candidate summaries according to the original target / .target.

[0073] The computer system 102 may be configured to execute a fourth method (LA- AR) incorporating the length approximation module 108 and automated revisions module 116. This method combines length approximation module with a cross-measure revision process. For a target length Lo in measure Mo, the computer system 102 first approximates the target length in a more controllable measure Mi to generate an initial summary. However, instead of revising toward the approximated target, the automated revisions module 116 recontextualize the revision within the original measure Mo, presenting the initial summary as if it were generated under Mi’s constraints. This measurement context manipulation allows subsequent revisions to directly optimize for the original target Lo.

[0074] The computer system 102 may be configured to execute a fifth method (SF-AR) incorporating the sample filtering module 114 and automated revisions module 116. This method combines sample filtering module 114 with the automated revisions module 116 process sequentially. The computer system first passes the input length control parameters and the input content directly to the summarization model to generate a first sample. The summarization model 104 generates N candidate summaries Si, ... , Syand select the optimal summary S* using the sample filtering criterion. If S* is non-compliant, the computer system 102 may transmit the non-compliant S* sample to the automated revisions module 116, as described before using S* as the starting point, potentially iteratively.

[0075] In some embodiments, if the revised summary is compliant with the target length (e.g., within a specified threshold), the system may output the summary as final output. The output may be rendered through an API, presented in a UI, or stored in a local or remote database for downstream use.

[0076] The computer system 102 may be configured to execute a sixth method (SR) incorporating the sampled revisions module 118. This method integrates sampling into each step of the revision process that may process a plurality of revised samples in a feedback loop rather than a sequential operation. For each revision step i, the summarization model 104 generates N candidate summaries based on the revision prompt:{Si, 1, ... , Si, \'J = LLM \ (7', Si-1, Ltarget) (11)The computer system 102 selects the sample, 5*, defined as the most accurately generated sample candidate length based on the input length constraints, and if S* still exceeds the compliance threshold e, it becomes the input for the next revision step.

[0077] The computer system 102 may evaluate the plurality of integrated methods based 171604215198.1Docket No. 242616PCTon a systematic evaluation of both the inherent length control capabilities of the Llama-3 -8B-Instruct model and potential improvements through the methods described herein, using the YTSEG dataset (Retkowski and Waibel, 2024). More than 5 million summaries were generated in this evaluation with the YTSEG dataset. The YTSEG dataset comprises 19,299 video transcripts and was selected based on its diversity in length, structure, and domain. The transcript lengths in this dataset range from a few hundred to over ten thousand words. This variability ensured a robust assessment of length controllability across heterogeneous inputs. In order to establish control parameters, the computer system 102 compares the plurality of approach against a naive prompting baseline, which does not employ any additional control mechanisms. As previously described, the plurality of methods executed by the computer system 102 are based on the five core length control modules which include the (1) length approximation module, (2) target adjustment module, (3) sample filtering module (s r [1, 8]), (4) automated revisions module (r e [0, 5]), and (5) various integrated combinations of the length control modules.

[0078] Figures 6A-6C show example prompt templates, according to various embodiments of the present invention. The system employs a consistent prompting strategy across all approaches by instructing the model 104 to generate summaries matching the specified lengths. The computer system 102 may utilize various prompt templates including a baseline prompt 602, response priming 604, and revision prompt 606. The prompt templates 602-606 may be configured based on the word length parameters, characters, or token-length-controlled summaries. Additionally, for the bullet point parameter, the system may employ the bullet symbol (e.g., dot) as an additional prefix in the response to implicitly define the typographical symbol to be used for the bullet-point format. The same symbol may then be used in the counting function (see 710 in Figure 7).Figure 6A further shows the baseline prompt 602 which may directly instruct the model in the user message only. Figure 6B shows the response priming prompt 604 that may restate the instruction by prefilling the response to improve instruction-following. Figure 6C shows the revision prompt 606 that may automate follow-up instructions to refine summaries that do not meet length constraints, with additional response priming.

[0079] Figure 7 shows a plurality of count functions that may be configured to measure the length of a generated summary, according to various embodiments of the present invention. The plurality of count functions include a first count function 702 that shows a word count function, a second count function 704 that shows a Token Count Function, a third count function 706 that shows a Character Count Function, a fourth count function 708 that 181604215198.1Docket No. 242616PCTshows a sentence count function, and a fifth count function 710 that shows a bullet point count function. For example, the first count function 702 may be configured to use a library such as natural language toolkit (NLTK) for word and sentence counts, where the token counts are determined based on the BPE tokenizer employed by LLaMA 3. The character counts are straightforwardly computed from the generated text, whereas the bullet point counts are by tallying bullet characters.

[0080] The computer system 102 may be configured to evaluate the performance of the length control methods (e.g., LA-SF, LA-TA, LA-TA-SF, LA-AR, SF-AR, SR) based on a plurality of length controllability metrics including: Exact Match (EM), Length Compliance (LC), Length Deviation (LD), Compression Rate (CR), Perplexity (PPL), and AlpacaEval. The EM metric may quantify the proportion of generated lengths that precisely matches the targets, while the LC metric may measure adherence within a specified tolerance. The LD metric may quantify the absolute difference between observed and target lengths, the CR metric may provide insight into data compression by comparing target and observed lengths, the PPL metric may be used to assess model uncertainty, by providing insight into generation quality, and the AlpacaEval metric may be used as an LLM-as-a-Judge metric to assess overall generation quality. The plurality of metrics may be based on the following equations, where N denotes the number of instances, Ly,i represents the observed length for the zthinstance and Ltrisignifies the target length.

[0081] The EM metric may quantify the proportion of the observed lengths Lyithat exactly match their target lengths Lt,i, as defined asEM provides a normalized measure of the frequency with which observed lengths precisely adhere to their target values, particularly useful for structural length measures.

[0082] The LC metric may quantify the proportion of the observed lengths Ly,i that are within a tolerance T of their target lengths Lt,i, as defined as<LC provides a normalized measure of adherence within a specified relative tolerance, offering a more flexible assessment of granular measures than EM by allowing slight deviations.

[0083] The LD metric may measure the average absolute difference between the observed lengths Ly,i and the target lengths Lt,i, as defined as191604215198.1Docket No. 242616PCTLD quantifies the average deviation from the target lengths. Unlike normalized measures, the LD metric reflects absolute errors, making it scale-sensitive and limiting comparability across varying length scales.

[0084] The CR metric may quantify the average ratio of the target length Lt,tto the observed lengths Ly,i as defined asThe RC metric provides insight into the relative compression of the target data by comparing the target length to the observed length, which serves as a descriptive statistic rather than an evaluation metric.

[0085] In some embodiments, metrics such as compression rate and perplexity may be primarily used in sample filtering modules to identify high-quality candidate summaries, whereas metrics such as exact match and length deviation may drive automated revision logic. Different modules may use the same base metric with distinct tolerance bands or weightings.

[0086] Figure 8 shows a table summarizing perplexity score (PPL) for length control parameters including characters, tokens, and words, according to various embodiments of the present invention. In this example, the perplexity scores were calculated with the pretrained LLaMA-3-8B language model. An average perplexity score may be calculated solely for length-compliant samples to minimize potential length bias and ensure fair comparison within a target.

[0087] In one example, the computer system 102 may be configured to evaluate model outputs with existing models such as GPT-4o (gpt-4o-2024-05-13) to evaluate model outputs following the AlpacaEval protocol and prompt. For each method and target length combination, the computer system 102 may select the 100 summary pairs where baseline and method outputs had the least length difference to minimize length bias. The total cost for evaluation was 51.73, which indicates the win rate as a percentage against the baseline model. Given the sample size per condition, individual cell values should be interpreted with consideration for sampling uncertainty, while averaged results across conditions provide reliable indicators of overall performance.

[0088] In the automated revisions module 116, the computer system 102 may set the revision trigger threshold e equal to the tolerance T used in the LC metric (±10% of target length). However, it’s worth noting that in practice, this threshold can be adjusted to limit revisions for summaries that excessively deviate from the length target, balancing accuracy201604215198.1Docket No. 242616PCTwith computational efficiency.

[0089] The limited maximum context length of LLaMA 3 (8,192 tokens) imposes constraints on the input prompt. Since we reserve at least 1,024 tokens for generation, only 7,168 tokens remain available for the input, which includes both the user instruction and the source document. Given that the dataset contains numerous lengthy documents, many of which exceed this limit, any necessary truncation is applied to the original source text to ensure it fits within the remaining context window.

[0090] Figures 9A-B show tables comparing the influence of response priming (RP) and sampling temperature (Temp.) on the length adherence of length control parameters, according to various embodiments of the present invention. In this example, the computer system evaluates the impact of response priming (RP) and sampling temperature r e {0.3, 0.7, 1.0} on length adherence and select RP values using the LLaMA3 model. Based on this evaluation, the computer system 102 is configured with the response priming and a temperature value of 0.7, based on their performance in preliminary tests. Figures 9A and 9B further show that incorporating RP significantly improves instruction-following and improves both LC and LD across different temperatures. For a target length of 50 words, LC improves from 75.6% without RP to 79.7% with RP at a temperature of 0.7. Lower sampling temperatures slightly enhance length control consistency, but the combination of RP and a temperature of 0.7 provides a good balance between adherence and output diversity.

[0091] As shown in Figure 5, there may be significant differences in length controllability across different length control parameters (e.g., measures). Notably, the summarization models may demonstrate near-perfect compliance for structural parameters, achieving high exact match rates for both sentence and bullet point counts. However, the evaluation of granular parameters like word count, token count, and character count have shown variability in performance. The summarization models may achieve relatively accurate length control over word counts but exhibit significantly lower compliance rates and greater deviations for token and character counts. These varying performances across granular parameters underscore the benefit of the length approximation module, which exploits these differences.

[0092] In various aspects, the summarization models may adapt the content density based on structural constraints, as seen in Figure 5 where the average sentence length decreases from 50.3 words for one-sentence summaries to 22.0 words for eight sentences. This pattern is mirrored in bullet point summaries, where the average length per bullet point decreases non-linearly from 51.6 words with three bullet points to 27.7 with eight. These trends indicate 211604215198.1Docket No. 242616PCTthat when constrained to fewer sentences or bullet points, the model packs more content into each, rather than simplifying. This contrasts with the relatively consistent sentence lengths observed in word, token, and character-controlled or qualitatively controlled summaries (see Figures 4 and 5).

[0093] Relatedly, both token-to-word and character-to-word ratios vary with summary length. For example, the token-to-word ratio decreases from 1.28 for 50-word summaries to 1.21 for 200-word summaries, while the character-to-word ratio drops from 6.39 to 6.15 (see Figure 5). These parallel trends provide strong evidence for shifts in lexical complexity as summary length increases.

[0094] Figure 10A shows a scatter plot 1002 of a plurality of data points representing generated summaries, according to various embodiments of the present invention. The scatter plot 1002 comprises a smoothed trendline for deviation and perfect accuracy with a highlighted tolerance of + / - 10%. The scatter plot 1002 illustrates a nonlinear relationship between target length (x-axis) and LD (y-axis), with adherence decreasing for longer summaries. A systematic bias towards under-generation emerges for targets exceeding -125 words. This trend is further supported by Figure 5, indicating an inverse correlation between target length and adherence, with compliance decreasing for longer targets (e.g., 8 sentences, 150-200 words, 8 BPs).

[0095] Conversely, Figure 10B shows a scatter plot 1004 of a plurality of data points representing generated summaries based on input words length (x-axis) in comparison to the length deviation of the generated summaries, according to various embodiments of the present invention. The scatter plot 1004 comprises a smoothed trendline for deviation and perfect accuracy. The scatter plot 1004 of generated summaries shows minimal correlation with between the input words length and LD, indicating robust length control across diverse document sizes. The scattered data points demonstrate flexibility in handling non-standard word targets. These findings highlight the model’s efficacy in producing specified-length summaries while identifying a length-dependent bias, motivating targeted improvements in handling longer summaries such as target adjustment.

[0096] As previously discussed in Figure 4, the summary request may comprise qualitative length qualifiers as input constraints for the generated summaries. Analysis of qualitative length quantifiers reveals the model’s attempt to adjust summary lengths accordingly (see Figure 4). These quantifiers yield a range of compression rates and output lengths, with “short” summaries averaging 127.2 ± 44.9 words (7.0 ± 4.8% CR) and “long” summaries reaching 343.0 ± 83.9 words (19.8 ± 13.5% CR). Notably, heteroscedasticity in 221604215198.1Docket No. 242616PCToutput metrics across quantifiers indicates inconsistent summarization control.

[0097] Figure 11 shows a scatter plot 1100 of a plurality of data points representing generated summaries, according to various embodiments of the present invention. The scatter plot 1100 summarizes the generated summary length and compliance rate against each input content length, for summaries generated using the “moderate” qualitative length quantifiers. The scatter plot 1100 comprises a trendline for summary length (smoothed) and compression rate (smoothed). Figure 11 further illustrates non-linear trends and variability in both summary length and compression rate relative to the input length for the generated summaries. This inconsistency between summary length and compression rate, combined with heteroscedastic results, suggests that qualitative length quantifiers provide imprecise control, limiting their effectiveness for length-controllable summarization tasks.

[0098] The computer system 102 may be configured for a plurality of different LLMs 104, with an expectation of similar results based on cross-model generalizability. The computer system 102 compares different LLaMA versions and other summarization models in zero-shot scenarios, highlighting that observed patterns of under-generation and overgeneration and preference for certain parameters are broadly consistent across architectures. This suggests that the length controllability challenges and model behaviors, identified in this evaluation, are not exclusive to a single model configuration. This reinforces the relevance of the identified results in length controllability in a broad range of summarization models. This comparative analysis across different model architectures reveals several key patterns in length controllability. This analysis includes measure-specific performance, input length sensitivity, target length effects, LLaMA model evolution, and model specific behaviors.

[0099] This analysis shows that generally all summarization models yield the highest length compliance performance when controlling output length at the word level. This is one example of how the system may be configured to implement specific methods, based on certain input length parameters, due to parameter-specific performance. For example, the LLaMA-based models, yields remarkable accuracy when controlling for word length.Specifically, the LLaMA-3.2-1 IB-Vision-Instruct achieves up to 96.4% compliance with a 100-word target. In contrast, character-level and token-level parameters yield significantly lower compliance rates, and thus highlighting the challenges for finer-grained parameters.

[0100] Additionally, the evaluation shows that most models are agnostic to the length of the input document as it applies to length compliance rates. Generally, the length of the input document has a minimal impact on the precision of their length control. However, in one example with gemma-2-9b-it, the model proves unstable when input exceeds about 3000231604215198.1Docket No. 242616PCTwords. In this example, the model enters a generation loop regardless of whether the input is truncated, underscoring a general and severe limitation in handling long inputs.

[0101] In some embodiments, the system may execute a control loop in which candidate summaries are iteratively generated and revised until one meets the specified compliance thresholds or a maximum number of iterations is reached. This loop may involve dynamic routing through one or more of the length control modules (e.g., LA^TA^AR). The control logic may maintain a state representation indicating the number of attempts, compliance history, and instruction provenance to prevent repetitive or divergent outputs.

[0102] In other examples with non-LLaMA models (e.g., Mistral-Nemo-Instruct-2407, gemma-2-9b-it, aya-expanse-8b), these models tend to over-generate summary content for relatively short target lengths. For example, at a 50-word target, the compliance rate may be as low as 14.5%, with some summaries substantially exceeding the requested length.However, as target lengths increase beyond approximately 125 words, these models, similar to LLaMA variants, begin to under-generate, often undershooting the requested length by tens of words. This pattern suggests a fundamental and broad-based difficulty for current language models in which the models struggle to maintain precise length control as output requirements become more substantial.

[0103] In recent LLaMA generations (e.g., v3.1, v3.2), these models consistently outperform their predecessors, especially for the word measure. Not only do these models achieve near-perfect compliance for shorter word targets, but they also exhibit more stable performance as target length grows. While non-LLaMA models generally lag behind in length control, some display distinctive strengths. For example, gemma-2-9b-it shows relatively strong token-level compliance (up to 35.5%), even though it struggles with longer input documents. These model-specific quirks, combined with the broader patterns observed, reinforce the inventors’ overall findings and add nuance to the understanding of how different architectures tackle the challenge of controlling output length.

[0104] Figures 12A-B show tables summarizing the performance evaluations for each length control approach, according to various embodiments of the present invention. The plurality of length control approaches were systematically evaluated to assess their impact on length controllability across different parameters (e.g., character length, token length). The length approximation approach significantly improved LC in measures where the model underperforms. For instance, in the character length measure, the baseline LC for a target of 150 characters was merely 7.0%. By approximating character counts using word counts, LC increased to 52.1% (see Figure 12A). Similar enhancements were observed in the token 241604215198.1Docket No. 242616PCTlength measure, where LC rose from 0.8% to 51.8% for a target of 50 tokens. However, we observe diminishing returns for longer targets as length adherence declines for longer summaries in the word measure.

[0105] The target adjustment approach showed notable effectiveness for longer targets, particularly where the model’s bias towards under-generation is pronounced. For instance, for a 200-word target, LC improved meaningfully from 26.8% to 49.5% after upward adjustment (Figure 12B). However, for shorter targets, such as 100 words, LC dropped from 84.0% to 61.7%. This suggests that refining the conversion model or introducing a threshold to activate adjustments only under substantial under-generation bias could help avoid performance degradation on shorter outputs.

[0106] Figures 13 A and 13B show tables of performance evaluation data for length approximation (LA) and target approximation (TA) based on length compliance (LC) and length deviation (LD) in comparison to baseline performance evaluation data, for the YTSEG dataset and the CNN / DM dataset, according to various embodiments of the present invention. The results in Figure 13 A show that the statistical mappings used for LA are robust and can be effectively applied to datasets beyond the one from which they were originally derived. Specifically, the LA method with coefficients derived from the YTSEG dataset significantly and similarly improved LC for the CNN / DM dataset. For example, in the character length parameter for the CNN / DM dataset, the LC increased from 24.0% in the baseline to 65.6% using LA for a target of 150 characters. This improvement suggests that the length approximation coefficients generalize well to another dataset with differences in content, style, and structure.

[0107] Figures 14A and 14B show line graph plots comparing length compliance rates versus the number of samples and revisions for a plurality of target length parameters, according to various embodiments of the present invention. Figure 14B further includes histograms showing the relative change (i.e., improvement, no change, and degradation) in compliance for a target length of 200 words, as a percentage for each number of revisions.

[0108] Figures 15A-C show evaluation results for the sample filtering module 114 as a function of sample counts on the character length, token length, and word length parameters, according to various embodiments of the present invention. The sample filtering module 114 may be particularly effective in controlling summary lengths when the model generates at least some compliant summaries, and may require the output distribution to cover the target length with nontrivial density. Figure 15A shows that for N= 3 samples, notable improvements are achieved if the model initially generates diverse enough outputs. However,251604215198.1Docket No. 242616PCTwhen the model has a strong bias toward noncompliance-such as extreme under- or over-generation-filtering adds little benefit. Figure 14A further shows that LC consistently improves as the number of generated samples (N) increases, though the gains diminish after a certain point.

[0109] Figures 16A-C show evaluation results for iterative automated revisions by the automated revisions module 116 as a function of sample counts on the character length, token length, and word length parameters, according to various embodiments of the present invention. The automated revisions module 116 may be particularly effective for longer target lengths across all parameters, where their initial length compliance is generally lower. Figures 16A-C show the results after a single revision, highlighting substantial improvements even when initial compliance is extremely poor scenarios where sample filtering often fails. For iterative revisions, Figure 14B shows compliance trends over multiple revisions. Notably, the statistics for 200-word summaries reveal that most summaries improve with each revision, with only a small percentage showing any degradation in LD. Moreover, unlike sample filtering, iterative revisions show fewer signs of diminishing returns, enhancing length compliance effectively with each iteration.

[0110] Figures 17A-C show evaluation results for sampled revisions as an integrated method as a function of sample counts on the character length, token length, and word length parameters, according to various embodiments of the present invention. The integration of different methods yields highly effective strategies for improving length adherence. Sampled revisions with N = 1 has proven very effective, particularly when the initial performance is reasonable. For cases involving iterative sampled revisions, near-perfect compliance can be achieved across all parameters, given sufficient iterations (see Figures 17C). In contrast, when the base performance is extremely poor, a different combination proves more effective: length approximation module 108 paired with sample filtering module 114. The length approximation module 108 helps ensure that the model generates at least some outputs close to the target, thereby making sample filtering significantly more impactful. Additionally, target adjustment can further improve compliance, especially with longer summaries.[oni] The plurality of methods / approaches methods preserve, and in some cases, enhance the overall quality of generated summaries by content summarization models. Since the methods do not modify the underlying parameters of the LLM model 104 itself, significantly alter prompts, or impose restrictive decoding constraints, the present invention provides only minimal impact on summary quality. This expectation is supported by the exemplary perplexity analysis shown in Figure 8, which shows no degradation, suggesting 261604215198.1Docket No. 242616PCTthat key content quality features such as coherence and fluency are preserved. The only notable difference is a lower perplexity in the LA approach, implying that summaries generated via word measure may achieve higher quality than those based on finer-grained measures like tokens or characters. This finding is further validated through AlpacaEval (see Figure 18), where GPT-40 as a judge ranks controlled summaries generated pursuant to embodiments of the present invention to comparable baseline outputs (SF: 50.3%, AR:50.8%, TA: 53.5%), with LA again achieving a notably higher win rate of 61.9%. Figure 18 shows the win rate percentage against the baseline for different measures and target lengths. The results in Figure 18 were obtained using GPT-4o as judge, evaluating 100 pairs per condition. These results confirm that the methods maintain or enhance content quality while substantially improving length control.

[0112] Various aspects of the present invention show a comprehensive analysis of length-controllable summarization using summarization models in zero-shot settings. These results reveal thatLLMs excel in structural length control parameters (e.g., sentences, bullet points) but struggle with granular length control, particularly for character and token counts. The computer system 102 addresses this issue through several approaches including: length approximation, target adjustment, sample filtering, and automated revisions. These approaches significantly enhance length compliance across various parameters without compromising summary quality, with integrated approaches yielding the most substantial improvements and, in some cases, achieving near-perfect compliance. These results advance the understanding of LLM behavior in controlled text generation and provide practical strategies for implementing precise length control.

[0113] This study aims to be a comprehensive evaluation of multiple length parameters or measures, targets, and control methods across a diverse dataset, and several key findings were validated through cross-model and cross-dataset experiments. The various approaches executed by the computer system 102 demonstrate significant improvements in length controllability, with approaches like length approximation and target adjustment being virtually cost-free.

[0114] Additional analysis for the present invention is shown in Figures 19-21.

[0115] Figures 19A-C show length approximation and target adjustment evaluations for the plurality of length parameters including character length, token length, word length, according to various aspects of the present invention.

[0116] Figures 20A-D show evaluations of target length deviation by comparing the word length deviation of generated summaries against input target lengths for a plurality of LLM 271604215198.1Docket No. 242616PCTmodels, according to various aspects of the present invention. The analysis of length deviation across different parameters and models with data is down sampled for visualization. The target summary lengths ranged from 25 to 200 words and included smooth trendlines using local regression.

[0117] Figures 20E-H show evaluations of target length deviation by comparing the word length deviation of generated summaries against input word length of the original content for a plurality of summarization models, according to various embodiments of the present invention. The analysis of length deviation across different parameters and models with data is down sampled for visualization. The target summary lengths ranged from 25 to 200 words and include smoothed trend line using local regression.

[0118] Figures 21 A-C show baseline performance comparisons across different models, according to various embodiments of the present invention. Figure 21 A shows the baseline performance comparison for character length control performance, Figure 2 IB shows the baseline performance comparison for token length control performance, and Figure 21C shows the baseline performance comparison for word length control performance.

[0119] Examples of the method according to various aspects of the present disclosure are provided below in the following numbered clauses. An aspect of the method may include any one or more than one, and any combination of, the numbered clauses described below.

[0120] In a first general aspect, therefore, the present invention is directed to a method for controlling content length of generated summaries by a content summarization model. The method comprises transmitting, by a first one or more processors of a computer system to a summarization model server system that is in communication with the computer system via an electronic data network, content summary instructions for summarizing source content, wherein the summarization model server system executes, by a second one or more processors, instructions stored in a non-transitory memory corresponding to the content summarization model; the computer system generates the content summary instructions based on the source content and a summary generation request received by the computer system; and the summary generation request specifies length control parameters for constraining a length of a candidate summary by the summarization model server system for the source content. The method further comprises receiving, by the computer system, from the summarization model server system, the candidate summary for the source content generated by the content summarization model in response to the content summary instructions, wherein the content summarization model is not trained on the candidate summary, the computer system does not have access to internals of the content summarization model, and 281604215198.1Docket No. 242616PCTthe content summarization model is used in a zero-shot configuration; transmitting, by the computer system to the summarization model server system, updated content summary instructions, wherein the computer system generates the updated content summary instructions based on the candidate summary and a plurality of length control modules stored in a non-transitory memory and executed by one or more processors of the computer system; receiving, by the computer system from the summarization model server system, an updated summary of the source content generated by the content summarization model; determining, by the computer system, whether an observed length of the updated summary is compliant with at least (a) the length control parameters and (b) one or more of a plurality of compliance metrics; and in response to determining that the observed length of the updated summary satisfies the length control parameters and at least one of the plurality of compliance metrics, outputting, by the computer system, the updated summary.

[0121] In a first embodiment of the first aspect, the length control parameters comprise one or more of sentence length, word length, token length, character length, bullet point count, percentage-based shortening relative to a length of the source content, or a combination thereof, and the summary generation request comprises target length values associated with the length control parameters. Additionally or alternatively, the first embodiment of the first general aspect further comprises (i) the percentage-based shortening length control parameter is translatable into equivalent target length values for one or more of the sentence length, word length, token length, character length, or bullet point count parameters; (ii) the target length values are integer values; and / or (iii) the method further comprises determining, by the computer system, one or more of the target length values are based on qualitative length quantifiers in the summary generation request, wherein the qualitative length quantifiers comprise short, concise, brief, moderate, medium-length, comprehensive, verbose, and long.

[0122] In a second embodiment of the first general aspect, the plurality of length control modules comprise a target adjustment module, a length approximation module, a sample filtering module, an automated revisions module, and a sampled revisions module.

[0123] In a third embodiment of the first general aspect, the method may further comprises the steps of transmitting, by the computer system, the length control parameters to the target adjustment module, based on the length control parameters, and wherein the summary generation request comprises target length values associated with the length control parameters; and determining, by the target adjustment module of the computer system, modified target length values based on a first multi-order polynomial and the target length 291604215198.1Docket No. 242616PCTvalues, wherein the content summary instructions are generated based on the modified target length values. Additionally, the method may further comprise the streps of transmitting, by the computer system, the candidate summary to the sample filtering module, wherein the candidate summary comprises a plurality of candidate samples; determining, by the sample filtering module of the computer system, a compliance rate of the plurality of candidate samples, wherein the compliance rate is determined based on observed parameter lengths for each of the plurality of candidate samples and the target length values associated with the length control parameters, and wherein the compliance rate is determined based on one or more of a plurality of metrics; selecting, by the sample filtering module of the computer system, a first candidate sample of the plurality of candidate samples, and wherein the first candidate sample has a highest compliance rate of the plurality of candidate samples.

[0124] In a fourth embodiment of the first general aspect, the method may alternatively comprise the steps of transmitting, by the computer system, the length control parameters to the length approximation module, based the length control parameters; determining, by the length approximation module of the computer system, approximate target length values of the length control parameters based on the length control parameters, the target length values, and a length approximation model, wherein the length approximation model is a first multiorder polynomial configured to determine a first approximate target length value for a first length control parameter based on a second length control parameter. Additionally or alternatively, the method may further comprise the streps of receiving, by the target adjustment module of the computer system, the approximate target length values of the length control parameters from the length approximation module; adjusting, by the target adjustment module of the computer system, the target length values to adjusted target length values based on a second multi-order polynomial and the approximate target length values, wherein the second multi-order polynomial is configured to account for length deviation between the target length values of the length control parameters and generated summary lengths; and generating, by the target adjustment module of the computer system, the content summary instructions based on the adjusted target length values.

[0125] In a fifth embodiment of the first general aspect, the method may further comprise the steps of transmitting, by the computer system, the length control parameters and the candidate summary, based on the length control parameters, to one or more of the automated revisions module, the sample filtering module, or a combination thereof, wherein the candidate summary is generated based on a first set of instructions of the content summary instructions, and wherein the first set of instructions comprises a processed target length 301604215198.1Docket No. 242616PCTvalues by one or more of: the length approximation module, the target adjustment module, or a combination thereof. Additionally or alternatively, the method may comprise the length control parameters wherein one or more of character length, token length, or a combination thereof. Additionally or alternatively, the method may comprise the steps of transmitting, by the computer system, the candidate summary to the automated revisions module; determining, by the automated revisions module of the computer system, a compliance rate of the candidate summary, wherein the compliance rate is determined based on the observed length of the candidate summary and the target length values associated with the length control parameters; determining, by the automated revisions module, the candidate summary is non-compliant based on a threshold deviation to the target length values associated with the length control parameters; and updating, by the automated revisions module, the content summary instructions based on the candidate summary, wherein the content summary instructions are updated until a maximum number of revisions are reached, or the candidate summary is compliant based on the threshold deviation. Additionally or alternatively, the method may comprise the steps of transmitting, by the computer system, the candidate summary to the sample filtering module, wherein the candidate summary comprises a plurality of candidate samples; determining, by the sample filtering module of the computer system, a compliance rate of the plurality of candidate samples, wherein the compliance rate is determined based on the observed length for each of the plurality of candidate samples and the target length values associated with the length control parameters, and wherein the compliance rate is determined based on one or more of the plurality of compliance metrics; selecting, by the sample filtering module of the computer system, a first candidate sample of the plurality of candidate samples, and wherein the first candidate sample has a highest compliance rate of the plurality of candidate samples.

[0126] In a sixth embodiment of the first general aspect, the length control parameters comprise structural parameters and granular parameters, wherein the granular parameters define summary length using fine-grained linguistic or technical units, and the structural parameters define the summary length using high-level textual structures. Additionally or alternatively, (i) the structural parameters comprise sentences, bullet points, or paragraphs, and the granular parameters comprise character length, token length, and word length; (ii) the plurality of compliance metrics comprise: exact match, length compliance rate, length deviation, compression rate, perplexity score, and AlpacaEval protocol; and / or (iii) the method further comprise the steps of comprising, in response to determining that the observed length of the updated summary does not satisfy the length control parameters, at 311604215198.1Docket No. 242616PCTleast one of the plurality of compliance metrics, or a combination thereof, revising, by the computer system, the candidate summary with the plurality of length control modules.Additionally or alternatively, the method comprises the steps of (i) determining, by the computer system, a first language of the source content, wherein the first language is different than a target language for the candidate summary; translating, by the computer system, the source content into the target language as translated source content; and updating the content summary instruction based on the translated source content; (ii) determining, by the computer system, a first language of the candidate summary, wherein the first language is different than a target language for the candidate summary; translating, by the computer system, the candidate summary into the target language as updated summary; determining, by the computer system, the observed length of the updated summary is not compliant with at least the length control parameters, one or more of the plurality of compliance metrics, or a combination thereof; and in response to determining that the observed length of the updated summary does not satisfy the length control parameters, at least one of the plurality of compliance metrics, or a combination thereof, revising, by the computer system, the candidate summary in the first language with the plurality of length control modules; and / or the summary generation request comprises one or more real-time content generation requests based on the source content, wherein the source content is dynamic source content comprising any one of an audio stream or a video stream. The candidate summary comprises shortened text for subtitles, transcripts, or captions.

[0127] In a second general aspect, therefore, the present invention is directed to a system for controlling content length of generated summaries by a content summarization model. The system comprises a summarization model server system comprising a first one or more processors coupled to a non-transitory memory storing instructions thereon and configured to execute the instructions corresponding to the content summarization model; and a computer system configured to communicate with the summarization model server system, wherein the computer system is further configured to receive a summary generation request associated with source content, wherein the summary generation request comprises length control parameters that constrain a length of a candidate summary by the content summarization model; generate content summary instructions based on the source content and the length control parameters; receive the candidate summary from the summarization model server system for the source content generated by the content summarization model in response to the content summary instructions, wherein the content summarization model is not trained on the candidate summary, the computer system does not have access to internals of the content 321604215198.1Docket No. 242616PCTsummarization model, and the content summarization model is used in a zero-shot configuration; transmit updated content summary instructions to the summarization model server system, wherein the computer system generates the updated content summary instructions based on the candidate summary and a plurality of length control modules comprising one or more processors and a non-transitory memory storing instructions that when executed perform operations associated with each of the plurality of length control modules; receive a new content summary from the content summarization model in response to the updated content summary instructions; determine an observed length of the new content summary is compliant with at least (a) the length control parameters and (b) one or more of a plurality of compliance metrics; and in response to a determination that the observed length of the new content summary is compliant, output the new content summary based on the source content.

[0128] In a first embodiment of the second general aspect, the length control parameters comprise one or more of sentence length, word length, token length, character length, bullet point count, or a combination thereof, and the summary generation request comprises target length values associated with the length control parameters.

[0129] In a second embodiment of the second general aspect, the plurality of length control modules comprise one or more of a target adjustment module, a length approximation module, a sample filtering module, an automated revisions module, a sampled revisions module, or a combination thereof.

[0130] In a third embodiment of the second general aspect, the plurality of compliance metrics comprise one or more of exact match, length compliance rate, length deviation, compression rate, perplexity score, AlpacaEval protocol, or a combination thereof.

[0131] The examples presented herein are intended to illustrate potential and specific implementations of the present invention. It can be appreciated that the examples are intended primarily for purposes of illustration of the invention for those skilled in the art. No particular aspect or aspects of the examples are necessarily intended to limit the scope of the present invention. Further, it is to be understood that the figures and descriptions of the present invention have been simplified to illustrate elements that are relevant for a clear understanding of the present invention, while eliminating, for purposes of clarity, other elements. While various embodiments have been described herein, it should be apparent that various modifications, alterations, and adaptations to those embodiments may occur to persons skilled in the art with attainment of at least some of the advantages. Persons skilled in the art will appreciate that recited operations therein may generally be performed in any 331604215198.1Docket No. 242616PCTorder. Also, although various operational flow diagrams are presented in a sequence(s), it should be understood that the various operations may be performed in other orders than those which are illustrated, or may be performed concurrently. The disclosed embodiments are therefore intended to include all such modifications, alterations, and adaptations without departing from the scope of the embodiments as set forth herein.

[0132] List of Full Citations:• Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python, 1st edition. O'Reilly, Beijing; Cambridge [Mass.]. OCLC: ocn301885973.• Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Kenneth Heafield. 2024a. Iterative Translation Refinement with Large Language Models. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pages 181-190, Sheffield, UK. European Association for Machine Translation (EAMT).• Yihan Chen, Benfeng Xu, Quan Wang, Yi Liu, and Zhendong Mao. 2024b.Benchmarking Large Language Models on Controllable Generation under Diversified Instructions. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16): 17808-17816. Number: 16.• Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad AL Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, ... , and Zhiwei Zhao. 2024. The Llama 3 Herd of Models. arXiv preprint. ArXiv:2407.21783.• Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S. Liang, and Tatsunori B. Hashimoto. 2023. AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback. Advances in Neural Information Processing Systems, 36:30039-30069.• Marcio Fonseca and Shay Cohen. 2024. Can Large Language Model Summarizers Adapt to Diverse Scientific Communication Goals? In Findings of the Association for Computational Linguistics ACL 2024, pages 8599-8618, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.• Tanya Goyal, Junyi Jessy Li, and Greg Durrett. 2022. News Summarization and Evaluation in the Era of GPT-3. arXiv preprint. ArXiv: 2209.12356 [cs],• Chiori Hori, Sadaoki Furui, Rob Malkin, Hua Yu, and Alex Waibel. 2002. Automatic summarization of English broadcast news speech. In Proceedings of the second international conference on Human Language Technology Research, pages 241-246.341604215198.1Docket No. 242616PCT• Renlong Jie, Xiaojun Meng, Lifeng Shang, Xin Jiang, and Qun Liu. 2024. Prompt-Based Length Controlled Generation with Multiple Control Types. In Findings of the Association for Computational Linguistics ACL 2024, pages 1067-1085, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.• Juseon-Do Juseon-Do, Jingun Kwon, Hidetaka Kamigaito, and Manabu Okumura. 2024. InstructCMP: Length Control in Sentence Compression through Instruction-based Large Language Models. In Findings of the Association for Computational Linguistics ACL 2024, pages 8980-8996, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.• Yuta Kikuchi, Graham Neubig, Ryohei Sasano, Hiroya Takamura, and Manabu Okumura. 2016. Controlling Output Length in Neural Encoder-Decoders. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1328-1338, Austin, Texas. Association for Computational Linguistics.• Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Remi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d'Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov,... , and Oriol Vinyals. 2022. Competition-Level Code Generation with AlphaCode. Science, 378(6624): 1092-1097. ArXiv:2203.07814 [cs].• Xun Liang, Hanyu Wang, Yezhaohui Wang, Shichao Song, Jiawei Yang, Simin Niu, Jie Hu, Dan Liu, Shunyu Yao, Feiyu Xiong, and Zhiyu Li. 2024. Controllable Text Generation for Large Language Models: A Survey. arXiv preprint. ArXiv:2408.12599 [cs], • Michael Xieyang Liu, Frederick Liu, Alexander J. Fiannaca, Terry Koo, Lucas Dixon, Michael Terry, and Carrie J. Cai. 2024. "We Need Structured Output": Towards User-centered Constraints on Large Language Model Output. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA2024, Honolulu, HI, USA, May 11-16, 2024, pages 10:1-10:9. ACM.• Yizhu Liu, Qi Jia, and Kenny Zhu. 2022. Length Control in Abstractive Summarization by Pretraining Information Selection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1 : Long Papers), pages 6885-6895, Dublin, Ireland. Association for Computational Linguistics.• Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and 351604215198.1Docket No. 242616PCTPeter Clark. 2023. Self-Refine: Iterative Refinement with Self-Feedback. Advances in Neural Information Processing Systems, 36:46534-46594.• Takuya Makino, Tomoya Iwakura, Hiroya Takamura, and Manabu Okumura. 2019. Global Optimization under Length Constraint for Neural Text Summarization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1039-1048, Florence, Italy. Association for Computational Linguistics.• Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, and Jianfeng Gao. 2023. Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback. arXiv preprint. ArXiv:2302.12813.• Xiao Pu, Mingqi Gao, and Xiaojun Wan. 2023. Summarization is (Almost) Dead. arXiv preprint. ArXiv:2309.09558 [cs] .• Zhiqiang Rao, Hengchao Shang, Jinlong Yang, Daimeng Wei, Zongyao Li, Jiaxin Guo, Shaojun Li, Zhengzhe Yu, Zhanglin Wu, Yuhao Xie, Bin Wei, Jiawei Zheng, Lizhi Lei, and Hao Yang. 2023. Length- Aware NMT and Adaptive Duration for Automatic Dubbing. In Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 138-143, Toronto, Canada (in-person and online). Association for Computational Linguistics.• Fabian Retkowski. 2023. The current state of summarization. In Andreas Sudmann, Anna Echterhblter, Markus Ramsauer, Fabian Retkowski, Jens Schrbter, and Alexander Waibel, editors, Beyond Quantity. Research with Subsymbolic Al, 1 edition, pages 291-312. transcript Verlag, Bielefeld, Germany.• Fabian Retkowski and Alexander Waibel. 2024. From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptions. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - Volume 1: Long Papers, St. Julian's, Malta, March 17-22, 2024, pages 406-419. Association for Computational Linguistics.• Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Wi eting, Nanyun Peng, and Xuezhe Ma. 2023. Evaluating Large Language Models on Controlled Generation Tasks. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3155-3168, Singapore. Association for Computational Linguistics.• Ashok Urlana, Pruthwik Mishra, Tathagato Roy, and Rahul Mishra. 2024.Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects - A 361604215198.1Docket No. 242616PCTSurvey. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1603-1623, Bangkok, Thailand. Association for Computational Linguistics.• Alex Waibel, Hartwig Steusloff, Rainer Stiefelhagen, and others. 2005. CHIL:Computers in the human interaction loop.• Alexander Waibel and Christian Fuegen. 2012. Simultaneous translation of open domain lectures and speeches.• Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2022. Generating Sequences by Learning to Self-Correct. arXiv preprint. ArXiv: 2211.00053.• Wen Xiao, Yujia Xie, Giuseppe Carenini, and Pengcheng He. 2023. ChatGPT-steered Editing Instructor for Customization of Abstractive Summarization. arXiv preprint.ArXiv:2305.02483 [cs],• Xianjun Yang, Yan Li, Xinlu Zhang, Haifeng Chen, and Wei Cheng. 2023. Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization. arXiv preprint.ArXiv:2302.08081 [cs],• Zhongyi Yu, Zhenghao Wu, Hao Zheng, Zhe Xuan Yuan, Jefferson Fong, and Weifeng Su. 2021. LenAtten: An Effective Length Controlling Unit For Text Summarization. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 363-370, Online. Association for Computational Linguistics.• Weizhe Yuan, Ilia Kulikov, Ping Yu, Kyunghyun Cho, Sainbayar Sukhbaatar, Jason Weston, and Jing Xu. 2024. Following Length Constraints in Instructions.• Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen Mckeown, and Tatsunori B. Hashimoto. 2024. Benchmarking Large Language Models for News Summarization. Transactions of the Association for Computational Linguistics, 12:39-57. Place: Cambridge, MA Publisher: MIT Press.• Klaus Zechner and Alex Waibel. 1998. Using Chunk Based Partial Parsing of Spontaneous Speech in Unrestricted Domains for Reducing Word Error Rate in Speech Recognition. COLING 1998 Volume 2: The 17th International Conference on Computational Linguistics.371604215198.1

Claims

1. Docket No. 242616PCTCLAIMSWhat is claimed is:

1. A method for controlling content length of generated summaries by a content summarization model, the method comprising:transmitting, by a first one or more processors of a computer system to a summarization model server system that is in communication with the computer system via an electronic data network, content summary instructions for summarizing source content, wherein:the summarization model server system executes, by a second one or more processors of the summarization model server system, instructions stored in a non-transitory memory, the instructions corresponding to the content summarization model;the computer system generates the content summary instructions based on the source content and a summary generation request received by the computer system; andthe summary generation request specifies length control parameters for constraining a length of a candidate summary by the summarization model server system for the source content;receiving, by the computer system, from the summarization model server system, the candidate summary for the source content generated by the content summarization model in response to the content summary instructions, wherein the content summarization model is not trained on the candidate summary, the computer system does not have access to internals of the content summarization model, and the content summarization model is used in a zero-shot configuration;transmitting, by the computer system to the summarization model server system, updated content summary instructions, wherein the computer system generates the updated content summary instructions based on the candidate summary and a plurality of length control modules, wherein the plurality of length control modules are stored in a non-transitory memory and executed by one or more processors of the computer system;receiving, by the computer system from the summarization model server system, an updated summary of the source content generated by the content summarization model;381604215198.1Docket No. 242616PCTdetermining, by the computer system, whether an observed length of the updated summary is compliant with at least (a) the length control parameters and (b) one or more of a plurality of compliance metrics; andin response to determining that the observed length of the updated summary satisfies the length control parameters and at least one of the plurality of compliance metrics, outputting, by the computer system, the updated summary.

2. The method of claim 1, wherein the length control parameters comprise one or more of sentence length, word length, token length, character length, bullet point count, percentage-based shortening relative to a length of the source content, or a combination thereof, and wherein the summary generation request comprises target length values associated with the length control parameters.

3. The method of claim 2, wherein the percentage-based shortening length control parameter is translatable into equivalent target length values for one or more of the sentence length, word length, token length, character length, or bullet point count parameters.

4. The method of claim 2, wherein the target length values are integer values.

5. The method of claim 2, further comprising:determining, by the computer system, one or more of the target length values are based on qualitative length quantifiers in the summary generation request, and wherein the qualitative length quantifiers comprise short, concise, brief, moderate, mediumlength, comprehensive, verbose, and long.

6. The method of claim 1, wherein the plurality of length control modules comprise a target adjustment module, a length approximation module, a sample filtering module, an automated revisions module, and a sampled revisions module.

7. The method of claim 6, further comprising:transmitting, by the computer system, the length control parameters to the target adjustment module, based on the length control parameters, and wherein the summary generation request comprises target length values associated with the length control parameters; and391604215198.1Docket No. 242616PCTdetermining, by the target adjustment module of the computer system, modified target length values based on a first multi-order polynomial and the target length values, wherein the content summary instructions are generated based on the modified target length values.

8. The method of claim 7, further comprising:transmitting, by the computer system, the candidate summary to the sample filtering module, wherein the candidate summary comprises a plurality of candidate samples; determining, by the sample filtering module of the computer system, a compliance rate of the plurality of candidate samples, wherein the compliance rate is determined based on observed parameter lengths for each of the plurality of candidate samples and the target length values associated with the length control parameters, and wherein the compliance rate is determined based on one or more of a plurality of metrics; selecting, by the sample filtering module of the computer system, a first candidate sample of the plurality of candidate samples, and wherein the first candidate sample has a highest compliance rate of the plurality of candidate samples.

9. The method of claim 6, further comprising:transmitting, by the computer system, the length control parameters to the length approximation module, based the length control parameters;determining, by the length approximation module of the computer system, approximate target length values of the length control parameters based on the length control parameters, the target length values, and a length approximation model, wherein the length approximation model is a first multi-order polynomial configured to determine a first approximate target length value for a first length control parameter based on a second length control parameter.

10. The method of claim 9, further comprising:receiving, by the target adjustment module of the computer system, the approximate target length values of the length control parameters from the length approximation module; andadjusting, by the target adjustment module of the computer system, the target length values to adjusted target length values based on a second multi-order polynomial and the approximate target length values, wherein the second multi-order polynomial is 401604215198.1Docket No. 242616PCTconfigured to account for length deviation between the target length values of the length control parameters and generated summary lengths; andgenerating, by the target adjustment module of the computer system, the content summary instructions based on the adjusted target length values.

11. The method of claim 9, further comprising:transmitting, by the computer system, the length control parameters and the candidate summary, based on the length control parameters, to one or more of the automated revisions module, the sample filtering module, or a combination thereof, wherein the candidate summary is generated based on a first set of instructions of the content summary instructions, and wherein the first set of instructions comprises a processed target length values by one or more of the length approximation module, the target adjustment module, or a combination thereof.

12. The method of claim 11, wherein the length control parameters comprise one or more of character length, token length, or a combination thereof.

13. The method of claim 11, further comprising:transmitting, by the computer system, the candidate summary to the automated revisions module;determining, by the automated revisions module of the computer system, a compliance rate of the candidate summary, wherein the compliance rate is determined based on the observed length of the candidate summary and the target length values associated with the length control parameters;determining, by the automated revisions module, the candidate summary is non-compliant based on a threshold deviation to the target length values associated with the length control parameters; andupdating, by the automated revisions module, the content summary instructions based on the candidate summary, wherein the content summary instructions are updated until a maximum number of revisions are reached, or the candidate summary is compliant based on the threshold deviation.

14. The method of claim 11, further comprising:transmitting, by the computer system, the candidate summary to the sample filtering 411604215198.1Docket No. 242616PCTmodule, wherein the candidate summary comprises a plurality of candidate samples; determining, by the sample filtering module of the computer system, a compliance rate of the plurality of candidate samples, wherein the compliance rate is determined based on the observed length for each of the plurality of candidate samples and the target length values associated with the length control parameters, and wherein the compliance rate is determined based on one or more of the plurality of compliance metrics;selecting, by the sample filtering module of the computer system, a first candidate sample of the plurality of candidate samples, and wherein the first candidate sample has a highest compliance rate of the plurality of candidate samples.

15. The method of claim 1, wherein the length control parameters comprise structural parameters and granular parameters, wherein the granular parameters define summary length using fine-grained linguistic or technical units, and the structural parameters define the summary length using high-level textual structures.

16. The method of claim 15, wherein the structural parameters comprise sentences, bullet points, or paragraphs, and the granular parameters comprise character length, token length, and word length.

17. The method of claim 1, wherein the plurality of compliance metrics comprise: exact match, length compliance rate, length deviation, compression rate, perplexity score, and AlpacaEval protocol.

18. The method of claim 1, further comprising, in response to determining that the observed length of the updated summary does not satisfy the length control parameters, at least one of the plurality of compliance metrics, or a combination thereof, revising, by the computer system, the candidate summary with the plurality of length control modules.

19. The method of claim 1, further comprising:determining, by the computer system, a first language of the source content, wherein the first language is different than a target language for the candidate summary; translating, by the computer system, the source content into the target language as translated source content; and421604215198.1Docket No. 242616PCTupdating the content summary instruction based on the translated source content.

20. The method of claim 1, further comprising:determining, by the computer system, a first language of the candidate summary, wherein the first language is different than a target language for the candidate summary; translating, by the computer system, the candidate summary into the target language as updated summary;determining, by the computer system, the observed length of the updated summary is not compliant with at least the length control parameters, one or more of the plurality of compliance metrics, or a combination thereof; andin response to determining that the observed length of the updated summary does not satisfy the length control parameters, at least one of the plurality of compliance metrics, or a combination thereof, revising, by the computer system, the candidate summary in the first language with the plurality of length control modules.

21. The method of claim 1, wherein the summary generation request comprises one or more real-time content generation requests based on the source content, wherein the source content is dynamic source content comprising any one of an audio stream or a video stream.

22. The method of claim 21, wherein the candidate summary comprises shortened text for subtitles, transcripts, or captions.

23. A system for controlling content length of generated summaries by a content summarization model, the system comprising:a summarization model server system comprising a first one or more processors coupled to a non-transitory memory storing instructions thereon and configured to execute the instructions corresponding to the content summarization model;a computer system configured to communicate with the summarization model server system, wherein the computer system is further configured to:receive a summary generation request associated with source content, wherein the summary generation request comprises length control parameters that constrain a length of a candidate summary by the content summarization model; generate content summary instructions based on the source content and the length control parameters;431604215198.1Docket No. 242616PCTreceive the candidate summary, from the summarization model server system, for the source content generated by the content summarization model in response to the content summary instructions, wherein the content summarization model is not trained on the candidate summary, the computer system does not have access to internals of the content summarization model, and the content summarization model is used in a zero-shot configuration;transmit an updated content summary instructions to the summarization model server system, wherein the computer system generates the updated content summary instructions based on the candidate summary and a plurality of length control modules, wherein the plurality of length control modules comprise one or more processors of the computer system and a non-transitory memory storing instructions that when executed by the one or more processor perform operations associated with each of the plurality of length control modules;receive a new content summary from the content summarization model in response to the updated content summary instructions;determine an observed length of the new content summary is compliant with at least (a) the length control parameters and (b) one or more of a plurality of compliance metrics; andin response to a determination that the observed length of the new content summary is compliant, output the new content summary based on the source content.

24. The system of claim 23, wherein the length control parameters comprise one or more of sentence length, word length, token length, character length, bullet point count, or a combination thereof, and wherein the summary generation request comprises target length values associated with the length control parameters.

25. The system of claim 23, wherein the plurality of length control modules comprise one or more of: a target adjustment module, a length approximation module, a sample filtering module, an automated revisions module, a sampled revisions module, or a combination thereof.

26. The system of claim 23, wherein the plurality of compliance metrics comprise one or more of: exact match, length compliance rate, length deviation, compression rate, perplexity441604215198.1Docket No. 242616PCTscore, AlpacaEval protocol, or a combination thereof.451604215198.1

Images