A Method and System for Predicting Output Length Based on Large Model Activation and Sampling Parameters

Abstract

This invention discloses a method and system for predicting output length based on large model activation and sampling parameters, aiming to solve the problem of accurately predicting the length of content generated by large models in existing technologies. This method constructs a dataset covering intermediate activation states and sampling parameters, utilizes an attention mechanism to filter key tokens, and designs a multi-source feature fusion architecture to achieve high-precision prediction of output length. This invention improves output length accuracy by 10% to 20% in 32K-length context prediction and can be widely applied to intelligent dialogue, text generation, and resource scheduling scenarios.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and natural language processing technology, and in particular relates to a method for generation optimization and resource scheduling of large-scale language models. Background Technology

[0002] With the widespread application of large language models (LLMs) in natural language generation tasks, the uncertainty of output length in practical deployments has become increasingly prominent. For example, in streaming generation services, the inability to predict the output length in advance can lead to insufficient memory allocation or wasted resources, and even service interruptions due to cache overflow. In batch processing systems, the lack of accurate output length estimation will severely affect the efficiency of task scheduling and load balancing, resulting in response latency and reduced system throughput. This problem will further amplify system instability, especially in high-concurrency scenarios. Furthermore, from a user experience perspective, the inability to provide accurate generation progress prompts will significantly impact the user's perceived service quality and interaction satisfaction, limiting the application of large models in demanding commercial scenarios.

[0003] Currently, the industry's methods for estimating LLM output length are still in a relatively rudimentary stage, lacking a systematic and generalizable technical framework. Existing methods mainly fall into two common categories:

[0004] The first type of method is a simple regression prediction based on the length of the input prompt. This method relies solely on the number of tokens in the input text to train a prediction model to predict the output length. However, this type of method completely ignores the semantic complexity, syntactic structure, and the model's internal understanding of the input, resulting in extremely limited prediction accuracy. Especially when the semantic connection between input and output is weak or the generation task is complex, such as in open-domain question answering, creative writing, or logical reasoning scenarios, this type of method often exhibits significant errors because it fails to capture the semantic-level generation logic. Furthermore, this method is extremely sensitive to changes in model structure, training data distribution, and task type, and lacks the necessary generalization ability.

[0005] The second category of methods is local prediction based on input truncation. This approach attempts to approximate the output length by truncating some tokens from the beginning or end of the input text. However, it fails to effectively capture long-distance dependencies and key semantic information across the entire text. Especially for tasks requiring long-context understanding, such as long document summarization, multi-turn dialogues, and code generation, key information in the input text may be distributed arbitrarily. Local truncation can lead to semantic loss or distortion, causing the prediction results to deviate significantly from the actual output. Furthermore, this method is particularly unstable when handling structurally complex and logically nested input texts, making reliable deployment in real-world systems difficult.

[0006] Besides the shortcomings of the two methods mentioned above in semantic understanding and structural modeling, existing technologies generally suffer from a significant drawback: they fail to adequately consider the substantial impact of sampling parameters on output length. In fact, during text generation, parameters such as temperature, kernel sampling threshold (top-p), Top-K sample count (top-k), and repetition penalty directly affect the randomness and diversity of the model's generation process, leading to significant differences in text length generated from the same input under different sampling strategies. For example, a higher temperature value increases uncertainty in the generation process, easily resulting in a significant increase in output length; while a stronger repetition penalty suppresses repeated generation, potentially shortening the output length accordingly. Because existing technologies lack the ability to model these key sampling parameters, they struggle to adapt to the varied and complex generation scenarios in real-world applications and cannot maintain stable prediction performance under dynamic parameter configurations.

[0007] It is worth noting that although some studies have attempted to model output length using statistical learning or lightweight neural networks, these methods mostly rely solely on surface features of the input side or are only customized for specific models or tasks, failing to fundamentally address core issues such as insufficient semantic understanding, incompatible sampling strategies, and weak long text modeling capabilities. Therefore, existing technologies generally suffer from limitations in practical applications, including large prediction bias, poor scenario adaptability, and high system overhead.

[0008] In summary, there is an urgent need in this field for a high-precision output length prediction scheme that can deeply integrate internal model state information, effectively extract key semantics from input text, and possess good adaptability to sampling parameters. Such a scheme needs to understand the generation logic from the perspective of the model's internal working mechanism, while also taking into account the impact of external parameter configurations, thereby providing reliable length prediction capabilities for large-scale language model systems to address the challenges of resource scheduling, system stability, and user experience encountered in practical deployments. Summary of the Invention

[0009] To fundamentally address the problem of accurately predicting the length of content generated by large models using existing technologies, and to promote the efficient and stable application of large models in real-world scenarios, this invention proposes an output length prediction method and system based on large model activation and sampling parameters. By analyzing the internal activation state of the large model (LLM) during forward propagation and combining it with externally controllable sampling parameters, an output length prediction model is constructed and trained. This prediction model is then used to achieve accurate and efficient prediction of the length of the model's output text.

[0010] To achieve the above-mentioned objectives, the present invention proposes the following technical solution:

[0011] In a first aspect, this invention proposes an output length prediction method based on large model activation and sampling parameters, comprising the following steps:

[0012] Step 1: Obtain historical data on the statistical distribution of multiple actual output lengths and the input text of the request to be predicted;

[0013] Step 2: Extract the input text, and filter key tokens based on attention scores during the forward propagation of the large model to construct a condensed context representation and generate intermediate layer activation states;

[0014] Step 3: Obtain the intermediate activation state of the large model, and intercept the sampling parameter combination configured by the user according to the forward propagation;

[0015] Step 4: Perform feature fusion on the intermediate activation state, the sampling parameter combination, and the key token information to obtain a comprehensive feature vector matrix;

[0016] Step 5: Construct a length prediction model. Use the actual output length distribution to train the length prediction model. The training optimization goal is to make the length probability distribution output by the prediction model infinitely close to the real length distribution. Input the comprehensive feature vector matrix obtained in Step 4 into the trained length prediction model. Perform length prediction through the prediction model, calculate the length prediction category distribution probability, and divide the output length into five mutually exclusive category intervals.

[0017] In some implementations, step 2, which involves filtering key tokens based on attention scores, specifically includes:

[0018] Step 2.1: When calculating attention during forward propagation, obtain the attention score matrix corresponding to the target intermediate layer;

[0019] Step 2.2: Calculate the average attention score for each input token across all attention heads;

[0020] Step 2.3: Sort all input tokens in descending order according to the average score, and select the top K tokens with the highest scores as key tokens;

[0021] Step 2.4: Extract the embedding vectors corresponding to the selected key tokens and concatenate them into a key context matrix;

[0022] In some implementations, the process of obtaining the intermediate activation states of the large model in step 3 specifically includes:

[0023] Step 3.1: The user initiates a generation request with specific sampling parameters, including temperature=0.7, top-p=0.9, top-k=1000, and repetition-penalty=1.1;

[0024] Step 3.2: The input text is fed into the large model for forward propagation. During the propagation process, the activation state output of the intermediate layers of the model is intercepted and extracted.

[0025] In some implementations, the feature fusion process in step 4 specifically includes:

[0026] Step 4.1: Map the sampling parameters into a one-dimensional sampling parameter vector through the Embedding layer;

[0027] Step 4.2: Expand the sampling parameter vector according to the dimensions of the key token matrix;

[0028] Step 4.3: Perform matrix addition on the key token context matrix and the expanded sampling parameter vector;

[0029] Step 4.4: Output the comprehensive feature matrix with fixed dimensions.

[0030] In some implementations, the length prediction process in step 5 specifically includes:

[0031] Step 5.1: Input the comprehensive feature matrix into the pre-trained length prediction model;

[0032] Step 5.2: Extract deep features using the Bert-Base-Uncased model;

[0033] Step 5.3: Extract the last hidden state of the CLS token;

[0034] Step 5.4: Input the hidden state into the multi-class classification and obtain the output length distribution;

[0035] Step 5.5: Calculate the predicted value based on the length interval corresponding to the category with the highest probability.

[0036] Secondly, this invention also proposes an output length prediction system based on large model activation and sampling parameters. The system comprises a large model acquisition and processing module and a prediction execution module. The large model acquisition and processing module is responsible for acquiring historical data of the statistical distribution of multiple actual output lengths and the input text of the prediction request, and for collecting intermediate activation states, sampling parameters, and historical data of the statistical distribution of multiple actual output lengths in real time. The prediction execution module is responsible for achieving high-precision prediction of the length distribution based on a deep neural network architecture.

[0037] The large model acquisition and processing module further includes a large model preprocessing submodule and a sampling parameter processing submodule. The large model preprocessing submodule is responsible for extracting the input text, filtering key tokens based on attention scores during the large model's forward propagation, constructing a condensed contextual representation, generating intermediate layer activation states, and accurately filtering key tokens from the input text and extracting deep semantic features based on an improved attention analysis algorithm. The sampling parameter processing submodule is responsible for transforming discrete numerical parameters into feature representations compatible with deep learning models.

[0038] The high-precision prediction execution module further includes a deep semantic understanding and feature refinement module, connected to the sampling parameter processing module, which is responsible for fusing the projected parameter features with the extracted semantic features, performing deep semantic understanding and refinement, and converting the one-dimensional parameter vector into a two-dimensional structure that matches the semantic feature matrix; and further includes a multi-class classification submodule, connected to the deep semantic understanding and feature refinement module, which is responsible for mapping the deep features to specific length category predictions.

[0039] In some implementations, the large model preprocessing submodule includes a Transformer block, which further includes the following two sub-layers:

[0040] The first group, consisting of a normalization layer-attention layer-normalization layer-multilayer perceptron structure, is responsible for calculating self-attention based on the large model input sequence. Specifically, it includes:

[0041] The first normalization layer normalizes the input text sequence and outputs a normalized token embedding vector.

[0042] The attention layer receives the output of the first normalization layer, calculates self-attention through query, key-value matrix operations, and generates attention scores and context-aware representations.

[0043] The second normalization layer normalizes the output of the attention layer;

[0044] The multilayer perceptron performs a nonlinear transformation on the output of the second normalized layer, and its output is connected with the input of the second normalized layer through a residual connection to serve as the final output of the first sublayer.

[0045] The second group of structures, consisting of a normalization layer-attention layer-normalization layer-multilayer perceptron, is responsible for further processing of the output from the first sub-layer. Specifically, it includes:

[0046] The first normalization layer normalizes the output of the first sub-layer;

[0047] The attention layer receives the normalized output and further refines the features through masked self-attention or cross-attention mechanisms.

[0048] The second normalization layer normalizes the output of the attention layer;

[0049] The multilayer perceptron performs the final nonlinear transformation, and its output serves as the final output of this Transformer block, as the input of the next Transformer block, or as the source of the activation states of intermediate layers.

[0050] In some implementations, the sampling parameter processing submodule is responsible for converting discrete numerical parameters into feature representations compatible with deep learning models, specifically including:

[0051] The parameter receiving and standardization unit is used to receive the user-configured combination of sampling parameters and perform numerical verification and standardization processing. The sampling parameters include the Temperature parameter, Top-p parameter, Top-k parameter, and Repetitionpenalty parameter.

[0052] The parameter embedding representation learning unit maps each sampled parameter into a high-dimensional feature vector through an embedding network containing fully connected layers and activation functions;

[0053] The parameter dimension alignment unit expands the sampled parameter vector into a two-dimensional structure that matches the semantic feature matrix through copying and transformation operations;

[0054] The multimodal feature fusion unit employs multiple fusion strategies to deeply fuse the expanded sampling parameter features with the key token information that has been filtered by attention.

[0055] In some implementations, the deep semantic understanding and feature refinement submodule is connected to the sampling parameter processing submodule and is responsible for implementing deep semantic understanding and refinement of multi-source features based on the BERT model, specifically including:

[0056] The input processing unit is used to convert the fused parameter feature matrix into an input format that the model can process through the embedding layer. The feature matrix contains both intermediate activation states from the large model and projected sampling parameter information.

[0057] The multi-layer feature extraction unit extracts deep semantic features step by step through a Bert-Based-Uncased model consisting of 12 Transformer layers. Each Transformer layer contains a self-attention mechanism and a feedforward neural network to capture semantic information at different levels.

[0058] The feature aggregation unit uses the final hidden state labeled [CLS] as the aggregated representation of the entire input sequence to extract global generation intent information from the complex feature matrix;

[0059] The output connection unit outputs the refined deep features to a multi-class classifier for final length and class prediction.

[0060] The intermediate activation state is obtained by intercepting the output tensor of a preset intermediate layer during the forward propagation of a large model.

[0061] In some implementations, the multi-class classification submodule is connected to the deep semantic understanding and feature refinement module, and is used to map deep features to specific length class predictions. Its network structure consists of multiple linear layers and ReLU activation functions connected in sequence, mapping the hidden state to the output length class probability distribution. The output length is divided into 5 mutually exclusive class intervals, and the predicted value is determined by the principle of maximizing probability.

[0062] Compared with existing technologies, this invention achieves the following significant beneficial effects through multi-level technological innovation:

[0063] 1) By utilizing the activation states of the intermediate layers of a large model, the deep understanding and generation intent of the model of the input semantics can be directly captured, avoiding the limitations of traditional methods based on surface features. Experiments show that in a 32K long context scenario, the prediction accuracy is improved by 10%-20%, thus significantly improving the prediction accuracy.

[0064] 2) The key token filtering mechanism based on attention scores can intelligently identify the core semantic units in the input text and filter redundant information, thereby enhancing the ability to process long texts; and, this mechanism ensures the condensed representation of the semantics of long texts during the dataset construction stage, improving prediction stability.

[0065] 3) The innovative feature fusion method supports embedded representations of parameters such as temperature and top-p, enabling the prediction system to flexibly adapt to various sampling strategies, thereby enhancing the adaptability of sampling parameters. Sampling parameter processing, as a key step in feature engineering, forms a complete technical loop with modules such as attention filtering and intermediate activation feature extraction. The processed parameter features provide important contextual information for the prediction model, enabling it to distinguish the length differences that may arise from the same input under different generation strategies. BERT-110M plays a dual role in this invention: semantic understanding and feature refinement. It not only deepens the understanding of input semantics but also transforms multi-source heterogeneous features into a unified representation with strong discriminative power. Enhanced system generalization ability: The design of the parameter projection layer enables the system to adapt to various combinations of sampling parameters, and BERT-110M's powerful representational capabilities ensure stable performance on texts from different domains.

[0066] 4) Accurate length prediction provides a reliable basis for memory allocation, task scheduling and streaming response, reduces the risk of service interruption, and improves system throughput in high-concurrency scenarios, thereby enabling the optimization of system resources.

[0067] 5) This method can be widely applied to application scenarios that require predicting the length of generated text, such as intelligent dialogue systems, automatic text generation, code completion, and real-time streaming response, and provides key technical support for resource allocation, task scheduling, and response optimization of model service systems.

[0068] 6) The modular design allows each component to be optimized independently, minimizing computational overhead while ensuring accuracy, thus meeting the needs of real-time prediction. Attached Figure Description

[0069] Figure 1 This is a flowchart of the output length prediction method based on large model activation and sampling parameters of the present invention;

[0070] Figure 2 This is a block diagram of the output length prediction system based on large model activation and sampling parameters of the present invention;

[0071] Figure 3 This is a roadmap for the output length prediction technique based on large model activation and sampling parameters of the present invention;

[0072] Figure 4 This is a schematic diagram illustrating the collection process of a large model output length distribution dataset used for training prediction in an embodiment of the present invention.

[0073] Figure 5 This is a schematic diagram of the prediction model training process in an embodiment of the present invention. Detailed Implementation

[0074] To more clearly illustrate the technical solutions and implementation methods of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. These drawings illustrate the core architecture, data processing flow, and model training method of the present invention, which helps those skilled in the art to better understand and implement the present invention.

[0075] Example 1: Figure 1 The overall flow of the output length prediction method based on large model activation and sampling parameters of the present invention is shown below, with specific steps as follows:

[0076] Step 1: Obtain historical data on the statistical distribution of multiple actual output lengths and the input text of the request to be predicted;

[0077] Step 2: Extract the input text, and filter key tokens based on attention scores during the forward propagation of the large model to construct a condensed context representation and generate intermediate layer activation states;

[0078] Step 3: Obtain the intermediate activation state of the large model, and intercept the sampling parameter combination configured by the user according to the forward propagation;

[0079] Step 4: Perform feature fusion on the intermediate activation state, the sampling parameter combination, and the key token information to obtain a comprehensive feature vector matrix;

[0080] Step 5: Construct a length prediction model. Use the actual output length distribution to train the length prediction model. The training optimization goal is to make the length probability distribution output by the prediction model infinitely close to the real length distribution. Input the comprehensive feature vector matrix obtained in Step 4 into the trained length prediction model. Perform length prediction through the prediction model, calculate the length prediction category distribution probability, and divide the output length into five mutually exclusive category intervals.

[0081] Following step 3, the above process further includes constructing a large model output length distribution prediction dataset, which includes intermediate layer activation states, attention score matrices, and historical data of statistical distributions of actual output lengths generated when the large model runs under various input texts and sampling parameter configurations.

[0082] Optionally, the process of filtering key tokens based on attention scores in step 2 specifically includes:

[0083] Step 2.1: When calculating attention during forward propagation, obtain the attention score matrix corresponding to the target intermediate layer;

[0084] Step 2.2: Calculate the average attention score for each input token across all attention heads;

[0085] Step 2.3: Sort all input tokens in descending order according to the average score, and select the top K tokens with the highest scores as key tokens;

[0086] Step 2.4: Extract the embedding vectors corresponding to the selected key tokens and concatenate them into a key context matrix.

[0087] Furthermore, the value of K can be 512, specifically limited by the maximum input length supported by the BERT model.

[0088] Furthermore, the sorting method can be an efficient sorting algorithm such as quicksort or heapsort.

[0089] Optionally, the process of obtaining the intermediate activation states of the large model in step 3 specifically includes:

[0090] Step 3.1: The user initiates a generation request with specific sampling parameters, including temperature=0.7, top-p=0.9, top-k=1000, and repetition-penalty=1.1;

[0091] Step 3.2: The input text is fed into the large model for forward propagation. During the propagation process, the activation state output of the intermediate layers of the model is intercepted and extracted.

[0092] Furthermore, the intermediate layer can be any intermediate layer of the model, such as the 7th layer or the 13th layer.

[0093] Furthermore, the dimension of the activation state depends on the model architecture and layer location, and is typically a high-dimensional tensor.

[0094] Optionally, the feature fusion process in step 4 specifically includes:

[0095] Step 4.1: Map the sampling parameters into a one-dimensional sampling parameter vector through the Embedding layer;

[0096] Step 4.2: Expand the sampling parameter vector according to the dimensions of the key token matrix;

[0097] Step 4.3: Perform matrix addition on the key token context matrix and the expanded sampling parameter vector;

[0098] Step 4.4: Output the comprehensive feature matrix with fixed dimensions.

[0099] Furthermore, feature fusion methods can include splicing, weighted summation, or cross-modal attention mechanisms.

[0100] Furthermore, the dimensions of the embedding layer can be adjusted according to specific application scenarios.

[0101] Optionally, the length prediction process in step 5 specifically includes:

[0102] Step 5.1: Input the comprehensive feature matrix into the pre-trained length prediction model;

[0103] Step 5.2: Extract deep features using the Bert-Base-Uncased model;

[0104] Step 5.3: Extract the last hidden state of the CLS token;

[0105] Step 5.4: Input the hidden state into the MultiClass Predictor to obtain the output length distribution;

[0106] Step 5.5: Calculate the predicted value based on the length interval corresponding to the category with the highest probability.

[0107] Furthermore, the length prediction model can be Bert-Base, Bert-Large, or other pre-trained models.

[0108] Furthermore, the MultiClass Predictor can be a combination of three linear layers and two ReLU activation functions.

[0109] Figure 2 and Figure 3 This paper illustrates a block diagram of the output length prediction system based on intermediate activation states and sampling parameters of a large model, and a technical roadmap for the output length prediction method and system embodiments based on intermediate activation states and sampling parameters of a large model. Figure 2 As shown, the system of the present invention includes a large model acquisition and processing module 100 and a prediction execution module 200. For example... Figure 3 As shown, the data flow and interaction relationships between the components highlight the technological innovations of this invention in multi-source feature extraction, deep fusion, and accurate prediction.

[0110] Specifically, the system architecture includes the following core components and processing flows:

[0111] The large model acquisition and processing module 100 is responsible for acquiring historical data of the statistical distribution of multiple actual output lengths and the input text of the request to be predicted. It collects intermediate activation states, sampling parameters and historical data of the statistical distribution of multiple actual output lengths in real time. These data are used to establish a complete data pipeline to ensure data quality and processing efficiency.

[0112] The prediction execution module 200 adopts a decoupled design of the deep semantic understanding and feature refinement (BERT-110M) submodule 300 and the multi-class classification submodule 220 (MultiClass Predictor). It is responsible for achieving high-precision prediction of length distribution based on the deep neural network architecture, supports horizontal expansion, can meet the performance requirements of applications of different scales, and at the same time ensures the high availability and stability of the service.

[0113] The large model acquisition and processing module 100 further includes a large model preprocessing submodule 110, which is responsible for extracting the input text, filtering key tokens based on attention scores during the forward propagation of the large model, constructing a condensed contextual representation, and generating intermediate layer activation states. Furthermore, based on an improved attention analysis algorithm, it accurately filters key tokens from the input text and extracts deep semantic features. This submodule integrates multiple natural language processing techniques, enabling it to adapt to different types of input text and ensuring the accuracy and robustness of feature extraction.

[0114] The large model preprocessing submodule 110 includes a Transformer block, which further includes the following two sub-layers:

[0115] User requests are propagated forward within the large model through a prefill process. During this process, the system extracts activation states from intermediate layers of the model (such as layers 7 and 13), which contain rich information about the model's deep understanding of the input semantics and its generation intent.

[0116] The first group consists of a normalization layer-attention layer-normalization layer-multilayer perceptron structure. This structure is the first sub-layer constituting the Transformer block and is mainly responsible for calculating self-attention based on the input sequence of the Large Language Model (LLM). A detailed description follows:

[0117] In the first normalization layer of the first sub-layer, the input data is the text sequence of prompt words in the Large Language Model (LLM). The input data is normalized through this layer to accelerate convergence, and the output is the token embedding vector (or tensor) of the normalized input text sequence of prompt words, whose dimension is consistent with the input.

[0118] In the first sub-layer of the attention layer, the input is the output of the first normalized layer. Its self-attention is calculated, i.e., through query, key-value matrix operations, the association strength between each token in the sequence and all other tokens is evaluated, generating an attention score, and based on this, a weighted context-aware representation is generated. The attention score serves as the basis for subsequently filtering the key tokens of the input text sequence of prompt words; the representation of each token incorporates global contextual information. This layer outputs a deeply encoded representation of the prompt word input text sequence after attention weighting and summing, with the same dimension as the input. During forward propagation, when the input text passes through a specific intermediate layer of the large model, the system intercepts and extracts the attention score matrix of that layer. This matrix contains information about the association strength between each token when the model understands the input semantics. After obtaining the original attention score matrix, the average attention score of each input token across all attention heads is calculated. This calculation process can be represented as:

[0119]

[0120] in, This represents the average attention score of the i-th input token. Let H represent the attention score of the i-th input token at the h-th attention head, where H represents the number of attention heads.

[0121] In the second normalization layer of the first sublayer, the input is the output of the attention layer. This input is then residually concatenated with the input of the attention layer (i.e., the output of the first normalization layer) and then fed into this normalization layer. The data is normalized again, and the output is the normalized tensor representing the deep-encoded prompt word input text sequence.

[0122] In the first sublayer of the multilayer perceptron, the input is the output of the second normalized layer, a feedforward neural network, typically containing two linear transformation layers and a nonlinear activation function. It performs independent and complex nonlinear transformations on the representation of each token to enhance the model's expressive power; the output is the transformed key token representation; this output is residually connected to the input of the second normalized layer, together serving as the final output of the first group and the input of the second group.

[0123] The second group consists of a normalization layer-attention layer-normalization layer-multilayer perceptron structure. This structure is the second sub-layer that makes up the Transformer block. Its structure is similar to the first group and it is responsible for the in-depth processing of the input text context corresponding to the key token.

[0124] In the first normalization layer of the second sublayer, the input is the final output from the first group. It has the same function as the first normalization layer in the first group: normalizing the input and outputting the normalized tensor represented by the transformed token.

[0125] The second sub-layer, the attention layer, receives the output of the first sub-layer. This layer is typically a masked self-attention layer or a cross-attention layer to ensure the autoregressive nature of the generation process. Attention calculations are performed again to further refine features, and the output is a context-aware representation of the input text sequence containing prompt words.

[0126] The second sublayer is the second normalized layer, which receives the output of the attention layer (usually with a residual connection to the input before normalization), performs normalization processing, and outputs a normalized tensor of the context-aware representation of the prompt word input text sequence.

[0127] The second sub-layer, the multilayer perceptron, receives the output from the second layer mentioned above, performs a final nonlinear transformation, and outputs the final output of this Transformer block. This output will serve as the input to the next Transformer block, or, if it is the last block, as the input to the model head (such as the language model head) to predict the next token. The activation states of intermediate layers are obtained from the output.

[0128] In summary, the forward propagation process of the large language model employs refined information processing through multiple cascaded Transformer blocks. Each "one-layer-attention-normalization-multilayer perceptron" structure is a powerful feature extraction unit, progressively transforming the original token embeddings into activation states rich in deep semantic information through layer-by-layer processing.

[0129] This invention achieves accurate prediction of the output length by intercepting and utilizing these key intermediate activation states in conjunction with sampling parameters.

[0130] The large language model acquisition and processing module 100 further includes the sampling parameter processing submodule 120, which is responsible for converting discrete numerical parameters into feature representations compatible with deep learning models, thereby realizing a complete sampling parameter processing flow for accurate modeling of externally controllable factors. The specific description is as follows:

[0131] i. Parameter Reception and Standardization: Receive user-configured combinations of sampling parameters, typically in key-value pairs. Perform numerical verification and standardization on each parameter to ensure its value falls within the preset effective range during model training. Key technical parameters involved include: Temperature parameter: controls generation randomness; higher values ​​result in more diverse outputs; Top-p parameter: sets the kernel sampling threshold, affecting the size of the candidate token set; Top-k parameter: limits the number of candidate tokens sampled in each step; Repetition penalty parameter: determines the degree to which repeated generation is suppressed.

[0132] ii. Perform parameter embedding representation learning: Sampled parameters are mapped into continuous feature vectors through a dedicated embedding layer. This step is one of the core innovations of this invention, addressing the modal discrepancy between numerical parameters and semantic features. Each sampled parameter is first processed through an independent embedding network. This network consists of trainable fully connected layers and appropriate activation functions (such as ReLU or GELU), mapping a single numerical value into a high-dimensional vector with rich semantic information. Embedding transformation process: [Scalar parameter] → [Fully connected layer] → [Activation function] → [High-dimensional feature vector]. This high-dimensional vector embedding representation not only includes the numerical information of the parameters but also learns the correlation between parameters and the potential influence patterns on the output length through the training process.

[0133] iii. Perform parameter dimension alignment and feature expansion: Since the sampled parameter vector and the semantic features (intermediate activation state, key token embedding) extracted from the large model may differ in dimension, the system designs an intelligent dimension alignment mechanism.

[0134] The sampling parameter vector is intelligently expanded according to the dimension of the key token matrix. Through copying and transformation operations, the one-dimensional parameter vector is expanded into a two-dimensional structure that matches the semantic feature matrix, preparing for subsequent fusion operations.

[0135] iv. Multimodal Feature Fusion: The expanded sampling parameter features are deeply fused with attention-filtered key token information. This invention proposes multiple fusion strategies to ensure effective interaction between features of different modalities.

[0136] The main integration methods include:

[0137] Matrix addition fusion: Parametric features and semantic features are directly added together to achieve information overlay.

[0138] Cross-modal attention mechanism: Allows parameter features and token features to pay attention to each other, learning deeper relationships.

[0139] Gated fusion network: dynamically adjusts the contribution of two types of features through learnable gating weights.

[0140] This invention is the first to systematically model the nonlinear effect of sampling parameters on output length. Through an end-to-end training process, the predictive model is able to learn complex patterns such as "high-temperature parameters typically lead to longer outputs".

[0141] Because the sampling parameter processing module is trained jointly with the prediction model, the system possesses the ability to generalize to unknown parameter combinations. Even when encountering parameter configurations not present in the training data, the model can make reasonable predictions based on the learned relationships between parameters. By designing the parameter processing as a lightweight embedded network, this invention minimizes computational overhead while maintaining prediction accuracy. The parameter processing process adds only a slight inference delay, making it suitable for real-time applications. Therefore, improved results with optimized computational efficiency are achieved.

[0142] Optionally, Figure 4 The dataset construction process is illustrated. Following step 3, a further step involves constructing a large model output length distribution prediction dataset. This dataset includes intermediate layer activation states, attention score matrices, and historical data on the statistical distribution of actual output lengths generated by the large model under various input text and sampling parameter configurations. During the training data collection phase, the system covers a broad sampling parameter space, providing a data foundation for the effective training of the parameter processing module. The dataset construction process specifically includes:

[0143] Input prompts were collected from various publicly available corpora, covering multiple types including question-and-answer, writing, translation, and code.

[0144] Configure multiple sampling parameter combinations for each prompt, covering the common range of each parameter;

[0145] For each (prompt, sampling parameter) pair, run the large model to generate it K times.

[0146] Record the activation of the intermediate layer, sampling parameters, key token information, and the actual output length distribution;

[0147] Furthermore, the number of repetitions K can be 50, 100, or 200 times.

[0148] Furthermore, the combination of sampling parameters should uniformly cover the value range of each parameter.

[0149] Therefore, the entire process of training data acquisition mainly includes the following key steps:

[0150] The first part, the input source section, demonstrates the process of collecting diverse texts from multiple public corpora, covering various types such as question-and-answer, writing, translation, and code generation, ensuring sufficient diversity in input texts in terms of length, semantic complexity, and stylistic features.

[0151] The second stage is the parameter configuration area, demonstrating how to configure various combinations of sampling parameters for each prompt. This includes setting parameters such as temperature, top-p, top-k, and repetition-penalty. It also shows how to set the max-tokens parameter based on the maximum generation length of the large model to ensure the output stops naturally. Additionally, to collect the request output distribution, we need to set the generation count M for each request (e.g., M=50).

[0152] The third stage involves inferring the request prompt using a large model. During the prefill phase of the inference process, it is necessary to preserve the intermediate activations of the model.

[0153] The fourth stage involves statistically analyzing the output lengths generated M times by the large model in Part 3, which yields the requested output length distribution. Furthermore, after obtaining the output length distribution for all data in the dataset, all output lengths can be divided into 5 categories using 5 quantiles, with each category corresponding to a length interval. For example, the llama3-8b-Instruct model supports a maximum context length of 8192. Sort the output lengths of all data in the dataset generated by the model, and take the 20%, 40%, 60%, and 80% quantiles after sorting: 400, 1152, 4600, and 7776, respectively. This results in the intervals being divided into: Interval Category 1 (0, 400), Interval Category 2 (400, 1152), Interval Category 3 (1152, 4600), Interval Category 4 (4600, 7776), and Interval Category 5 (7776, 8192).

[0154] Finally, the training predictor dataset can be obtained through the above process, which includes the intermediate activation states of the model, sampling parameters, and output distribution.

[0155] Optionally, Figure 5 The model training process is illustrated. During training, the sampled parameters serve as crucial input features in model optimization, ensuring the robustness and accuracy of the final prediction system in real-world deployments. The process specifically includes:

[0156] The comprehensive feature matrix extracted from the dataset will be used as the input feature X;

[0157] Use the actual output length distribution as the training label Y;

[0158] KL divergence or mean square error is used as the loss function;

[0159] The AdamW optimizer is used to iteratively optimize the model parameters;

[0160] Monitor the loss changes on the validation set until the model converges;

[0161] Furthermore, the optimizer can be Adam, SGD, or other gradient descent algorithms.

[0162] Furthermore, an early stopping strategy can be used during training to prevent overfitting.

[0163] By employing the multi-source feature fusion method based on intermediate activation states and sampling parameters, this invention effectively captures the deep generation intent of large models, significantly improving the accuracy and stability of output length prediction. This method enhances its ability to process long texts through a key token filtering mechanism and strengthens its adaptability under different sampling strategies through parameter fusion design, thus providing reliable technical support for resource optimization, memory management, and user experience improvement in large model inference services.

[0164] Practical application verification: In diverse testing scenarios, this sampling parameter processing mechanism demonstrates significant technical advantages. Especially when parameter configurations vary considerably, prediction accuracy is improved by 15-25% compared to traditional methods that ignore sampling parameters. This improvement directly translates into increased system resource utilization and a better user experience.

[0165] Through the complete implementation of the above technical solution, this invention successfully solves the industry problem of sampling parameter adaptability and provides reliable technical support for the intelligent operation and maintenance of large model service systems.

[0166] The parameter projection layer is one of the core innovative modules of this invention, mainly addressing the modal discrepancy between numerical sampling parameters and semantic features. Traditional methods directly concatenate numerical parameters to feature vectors, failing to effectively model the complex relationships between parameters and their nonlinear impact on output length. Dimension alignment mechanism:

[0167] Category terms represent the set of key tokens selected through an attention mechanism.

[0168] Hidden layer states represent the intermediate layer activation states generated during the forward propagation of a large model, and are the key carriers for capturing the generation intent within the model.

[0169] The delimiter represents implicit semantic separation achieved through attention mechanisms and the model's internal structure. Its core function is to divide and aggregate input information. This includes key token filtering: based on attention scores, the top K key tokens are selected from the input sequence. These tokens naturally divide the input content into "core semantic units" and "redundant information," achieving dynamic separation. The special role of the CLS token: during the prediction phase, the [CLS] token in the BERT model acts as a global semantic aggregator. Its final hidden state carries a compressed representation of the entire input sequence, functionally similar to a delimiter, used to output classification decisions.

[0170] The high-precision prediction execution module 200 further includes a deep semantic understanding and feature refinement (BERT-110M) module 210, which is connected to the sampling parameter processing module 120. It is responsible for fusing the projected parameter features with the extracted semantic features, performing deep semantic understanding and refinement, and converting the one-dimensional parameter vector into a two-dimensional structure that matches the semantic feature matrix.

[0171] The Deep Semantic Understanding and Feature Refinement (BERT-110M) module 210 operates based on the standard Transformer encoder architecture and is specifically optimized for the output length prediction task.

[0172] The specific process is as follows:

[0173] Input processing stage: The fused parameter feature matrix first passes through the embedding layer, transforming it into a processable input format. This matrix simultaneously contains intermediate activation states from the larger model and projected sampling parameter information.

[0174] Multi-layer feature extraction: Input embedding → 12-layer Transformer encoder → hidden states of each layer, extracting features step by step through the 12-layer Transformer encoder. Each layer includes a self-attention mechanism and a feedforward neural network, capable of capturing semantic information at different levels. Token feature aggregation: Specifically, the final hidden state labeled [CLS] is used as an aggregated representation of the entire input sequence. This design enables the model to extract global generative intent information from a complex feature matrix.

[0175] The multi-class classifier makes the final length category prediction based on refined features.

[0176] Obtaining the hidden layer state further includes:

[0177] Interception timing: When user input text propagates forward through a large model (such as LLaMA, Qwen, etc.), the system will intercept and extract the activation state from a preset intermediate layer (such as the 7th or 13th layer).

[0178] The output tensor of a specific layer can be captured in real time using hook functions or model interception techniques. This state is a high-dimensional vector, the dimensions of which depend on the model architecture and typically contain sequence length and hidden dimension information.

[0179] The hidden layer states encode the model's deep understanding of the input semantics, such as syntactic structure, logical relationships, and generation tendencies, providing a direct basis for length prediction.

[0180] The extracted hidden layer states are fused with sampling parameters and key token information to form a comprehensive feature vector. This process involves dimensionality reduction and standardization to ensure compatibility with downstream prediction models.

[0181] The multi-class classification submodule 220 is connected to the deep semantic understanding and feature refinement (BERT-110M) module 210 and is responsible for mapping deep features to specific length class predictions.

[0182] The network structure upon which this module is based consists of three linear layers combined with two ReLU activation functions: Input → Linear Layer → ReLU Activation → Linear Layer → ReLU Activation → Linear Layer → Softmax Normalization. This three-layer design enables the intermediate layers to incorporate non-linear transformation capabilities using ReLU activation functions. Finally, the output layer uses the Softmax function to convert the raw scores into a probability distribution of the categories. Category partitioning strategy: Based on the output length distribution of the training dataset, the system divides the possible output lengths into five mutually exclusive category intervals. This design ensures both the granularity of prediction and avoids the instability inherent in regression tasks.

[0183] Specifically, the input dimension is: 768-dimensional → linear layer (768→512) → ReLU activation → linear layer (512→256) → ReLU activation → linear layer (256→5) → Softmax normalization.

[0184] The length prediction model further includes a classification decision mechanism and optimization objective:

[0185] The final decision is made based on the principle of maximum probability: Predicted category = argmax(Softmax(final hidden state));

[0186] During training, the optimization objective is to minimize the KL divergence between the predicted distribution and the true distribution, and all parameters are optimized using the backpropagation algorithm.

[0187] The aforementioned sampling parameter processing module 120, deep semantic understanding and feature refinement (BERT-110M) module 210, and multi-class classification module 220 work together. The sampling parameter processing module 120 converts the sampling parameters into a feature representation that the model can understand through a parameter projection layer. The deep semantic understanding and feature refinement module (BERT-110M) 210 performs deep semantic extraction and refinement on the fused features. The multi-class classification module 220 makes the final length category prediction based on the refined features.

[0188] Specific embodiments of the present invention are as follows:

[0189] The first phase begins by acquiring raw training samples from a pre-constructed dataset, including intermediate activation states and attention score matrices. This data originates from a large-scale, diverse corpus covering text types of varying domains and complexity, ensuring the representativeness and comprehensiveness of the training data.

[0190] The second stage involves a key token filtering step based on the obtained attention score matrix. By calculating the average attention score of each input token across all attention heads, these tokens are sorted according to their scores, and the top K most important tokens are selected to form a condensed contextual semantic representation. This step effectively preserves the core information of the input text and filters out redundant content.

[0191] The third stage involves feature fusion and model forward propagation. Intermediate activation states, selected key token information, and corresponding sampling parameters are input into the feature fusion module. Through embedding layers and matrix operations, a unified comprehensive feature representation is generated. This feature representation is then fed into the prediction model, processed through multiple neural networks, and finally outputs the predicted request length probability distribution.

[0192] The fourth stage, after obtaining the predicted distribution, proceeds to the loss calculation stage. The predicted output length distribution is compared with the true length distribution labels, and either KL divergence or mean squared error is used as the loss function to quantify the degree of difference between the predicted results and the true values. This step provides a clear direction for model parameter optimization.

[0193] The fifth stage involves updating the model parameters. Based on the calculated loss value, the gradients of the parameters at each layer are calculated using the backpropagation algorithm, and all trainable parameters of the prediction model are updated using a gradient descent optimization algorithm (such as AdamW). This step enables the model to continuously learn and improve from the training data.

[0194] Finally, after updating the parameters, the process will return to the initial data acquisition step and begin a new training cycle. This iterative process will continue until the model's performance on the validation set reaches the convergence criterion, or until the preset number of training epochs is reached.

[0195] The entire training process forms a complete closed-loop optimization system, which gradually improves the accuracy and stability of the prediction model through multiple iterations. The steps are closely linked, forming an end-to-end supervised learning pipeline, ensuring that the predictor can fully learn the complex mapping relationship from input features to output length.

[0196] By employing the multi-source feature fusion method based on intermediate activation states and sampling parameters, this invention effectively captures the deep generation intent of large models, significantly improving the accuracy and stability of output length prediction. This method enhances its ability to process long texts through a key token filtering mechanism and strengthens its adaptability under different sampling strategies through parameter fusion design, thus providing reliable technical support for resource optimization, memory management, and user experience improvement in large model inference services.

[0197] In summary, this innovative architectural design provides a reliable technical foundation for the intelligent operation and maintenance of large model service systems, and effectively solves the industry problem of output length prediction.

[0198] First, a dedicated dataset is constructed to characterize the output length distribution of large models. This dataset comprehensively includes the activation states of intermediate layers, various sampling parameter configurations, and corresponding output lengths across multiple iterations. To improve prediction accuracy, this invention proposes three key technologies: First, it utilizes the intermediate layer activations generated during the forward computation of large models as the main input to the predictor, thereby capturing the internal state information of the model; second, it introduces a key token filtering mechanism based on attention scores, which can efficiently extract contextual information that retains core semantic information from long input sequences, ensuring the stability and accuracy of predictions under long text inputs; furthermore, this invention designs a general method for fusing sampling parameters and intermediate activation features, which can flexibly adapt to various common sampling strategies, significantly enhancing the practicality and accuracy of the predictor in real-world, complex large model application scenarios. This invention improves the accuracy of output length prediction by 10% to 20% under 32K contextual conditions.

[0199] To summarize again, the present invention includes the following core steps:

[0200] (1) Construct a large model output length distribution dataset

[0201] A large-scale model output length distribution dataset was constructed by systematically collecting diverse input texts of varying domains, complexities, and lengths. The target large-scale model was run under a wide range of sampling parameter configurations, comprehensively recording key data including intermediate layer activation states, complete sampling parameter configurations, attention score matrices, and the statistical distribution of output text lengths obtained from multiple repeated generation experiments. The dataset construction process particularly emphasized the spatial coverage of sampling parameters, ensuring that key parameters such as temperature, top-p, top-k, and repetition-penalty were fully sampled within their commonly used ranges, providing a high-quality data foundation for model training.

[0202] (2) Establish a key token filtering module based on attention mechanism

[0203] By deeply analyzing the attention score matrix of the model's intermediate layers, the importance score of each input token on different attention heads is calculated, establishing a multi-dimensional evaluation system to select the subset of key tokens that have the greatest influence on the generation process. This mechanism is specifically optimized for long text input scenarios, intelligently identifying core semantic units in the input text, effectively filtering redundant information, achieving efficient condensation of contextual semantic information, and significantly improving the stability and accuracy of long text prediction.

[0204] (3) Design a multi-layered feature fusion architecture

[0205] This module employs advanced cross-modal feature processing techniques, including intelligent dimensionality reduction and standardization of intermediate layer activation states, mapping sampling parameters into continuous feature vectors through embedded representation learning, deep encoding and semantic aggregation of key token information, and finally effectively integrating three types of heterogeneous features into a unified comprehensive feature representation through various techniques such as feature concatenation, weighted fusion, and cross-modal attention mechanisms, forming a feature space with strong discriminative capabilities.

[0206] (4) Construct a length prediction model based on deep neural networks

[0207] This model takes the fused feature vector as input and, through a carefully designed network structure, including advanced techniques such as multilayer perceptrons or Transformer encoders, calculates the probability distribution of the output length. The model employs an end-to-end training approach, optimizing the loss function to ensure prediction accuracy. Finally, based on the length interval corresponding to the class with the highest probability, it combines multiple calculation strategies to obtain accurate prediction results.

[0208] By implementing the above-mentioned technical solution, this invention effectively solves the key technical problem in predicting the output length of large models, and provides reliable technical support for the performance optimization and resource management of large models in practical applications.

[0209] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting output length based on large model activation and sampling parameters, characterized in that, Includes the following steps: Step 1: Obtain historical data on the statistical distribution of multiple actual output lengths and the input text of the request to be predicted; Step 2: Extract the input text, and filter key tokens based on attention scores during the forward propagation of the large model to construct a condensed context representation and generate intermediate layer activation states; Specifically, it includes: Step 2.1: When calculating attention during forward propagation, obtain the attention score matrix corresponding to the target intermediate layer; Step 2.2: Calculate the average attention score for each input token across all attention heads; Step 2.3: Sort all input tokens in descending order according to the average score, and select the top K tokens with the highest scores as key tokens; Step 2.4: Extract the embedding vectors corresponding to the selected key tokens and concatenate them into a key context matrix; Step 3: Obtain the intermediate activation state of the large model, and intercept the sampling parameter combination configured by the user according to the forward propagation; Step 4: Perform feature fusion on the intermediate activation state, the sampling parameter combination, and the key token information to obtain a comprehensive feature vector matrix; Step 5: Construct a length prediction model. Train the length prediction model using the actual output length distribution, aiming to make the length probability distribution output by the prediction model infinitely approximate the true length distribution. Input the comprehensive feature vector matrix obtained in Step 4 into the trained length prediction model. The model then predicts the length, calculates the probability distribution of the predicted length category, and divides the output length into five mutually exclusive category intervals. Specifically, this includes: Step 5.1: Input the comprehensive feature matrix into the pre-trained length prediction model; Step 5.2: Extract deep features using the Bert-Base-Uncased model; Step 5.3: Extract the last hidden state of the CLS token; Step 5.4: Input the hidden state into multi-class classification to obtain the output length distribution; Step 5.5: Calculate the predicted value based on the length interval corresponding to the category with the highest probability.

2. The output length prediction method based on large model activation and sampling parameters as described in claim 1, characterized in that, Step 3, the process of obtaining the intermediate activation states of the large model, specifically includes: Step 3.1: The user initiates a generation request with specific sampling parameters, including temperature=0.7, top-p=0.9, top-k=1000, and repetition-penalty=1.1; Step 3.2: The input text is fed into the large model for forward propagation. During the propagation process, the activation state output of the intermediate layers of the model is intercepted and extracted.

3. The output length prediction method based on large model activation and sampling parameters as described in claim 1, characterized in that, The feature fusion process in step 4 specifically includes: Step 4.1: Map the sampling parameters into a one-dimensional sampling parameter vector through the Embedding layer; Step 4.2: Expand the sampling parameter vector according to the dimensions of the key token matrix; Step 4.3: Perform matrix addition on the key token context matrix and the expanded sampling parameter vector; Step 4.4: Output the comprehensive feature matrix with fixed dimensions.

4. An output length prediction system based on large model activation and sampling parameters that performs the method described in any one of claims 1 to 3, characterized in that, The system includes a large model acquisition and processing module and a prediction execution module. The large model acquisition and processing module is responsible for acquiring historical data of the statistical distribution of multiple actual output lengths and the input text of the prediction request, and collecting intermediate activation states, sampling parameters and historical data of the statistical distribution of multiple actual output lengths in real time. The prediction execution module is responsible for achieving high-precision prediction of length distribution based on a deep neural network architecture. The large model acquisition and processing module further includes a large model preprocessing submodule and a sampling parameter processing submodule. The large model preprocessing submodule is responsible for extracting the input text, filtering key tokens based on attention scores during the large model's forward propagation, constructing a condensed contextual representation, generating intermediate layer activation states, and accurately filtering key tokens from the input text and extracting deep semantic features based on an improved attention analysis algorithm. The sampling parameter processing submodule is responsible for transforming discrete numerical parameters into feature representations compatible with deep learning models. The high-precision prediction execution module further includes a deep semantic understanding and feature refinement module, connected to the sampling parameter processing module, which is responsible for fusing the projected parameter features with the extracted semantic features, performing deep semantic understanding and refinement, and converting the one-dimensional parameter vector into a two-dimensional structure that matches the semantic feature matrix; and further includes a multi-class classification submodule, connected to the deep semantic understanding and feature refinement module, which is responsible for mapping the deep features to specific length category predictions.

5. The output length prediction system based on large model activation and sampling parameters as described in claim 4, characterized in that, The large model preprocessing submodule includes a Transformer block, which further includes the following two sub-layers: The first group, consisting of a normalization layer-attention layer-normalization layer-multilayer perceptron structure, is responsible for calculating self-attention based on the large model input sequence. Specifically, it includes: The first normalization layer normalizes the input text sequence and outputs a normalized token embedding vector. The attention layer receives the output of the first normalization layer, calculates self-attention through query, key-value matrix operations, and generates attention scores and context-aware representations. The second normalization layer normalizes the output of the attention layer; The multilayer perceptron performs a nonlinear transformation on the output of the second normalized layer, and its output is connected with the input of the second normalized layer through a residual connection to serve as the final output of the first sublayer. The second group of structures, consisting of a normalization layer-attention layer-normalization layer-multilayer perceptron, is responsible for further processing of the output from the first sub-layer. Specifically, it includes: The first normalization layer normalizes the output of the first sub-layer; The attention layer receives the normalized output and further refines the features through masked self-attention or cross-attention mechanisms. The second normalization layer normalizes the output of the attention layer; The multilayer perceptron performs the final nonlinear transformation, and its output serves as the final output of this Transformer block, as the input of the next Transformer block, or as the source of the activation states of intermediate layers.

6. The output length prediction system based on large model activation and sampling parameters as described in claim 4, characterized in that, The sampling parameter processing submodule is responsible for converting discrete numerical parameters into feature representations compatible with deep learning models, specifically including: The parameter receiving and standardization unit is used to receive the user-configured combination of sampling parameters and perform numerical verification and standardization processing. The sampling parameters include the Temperature parameter, Top-p parameter, Top-k parameter, and Repetition penalty parameter. The parameter embedding representation learning unit maps each sampled parameter into a high-dimensional feature vector through an embedding network containing fully connected layers and activation functions; The parameter dimension alignment unit expands the sampled parameter vector into a two-dimensional structure that matches the semantic feature matrix through copying and transformation operations; The multimodal feature fusion unit employs multiple fusion strategies to deeply fuse the expanded sampling parameter features with the key token information that has been filtered by attention.

7. The output length prediction system based on large model activation and sampling parameters as described in claim 4, characterized in that, The deep semantic understanding and feature refinement submodule, connected to the sampling parameter processing submodule, is responsible for implementing deep semantic understanding and refinement of multi-source features based on the BERT model, specifically including: The input processing unit is used to convert the fused parameter feature matrix into an input format that the model can process through the embedding layer. The feature matrix contains both intermediate activation states from the large model and projected sampling parameter information. The multi-layer feature extraction unit extracts deep semantic features step by step through a Bert-Based-Uncased model consisting of 12 Transformer layers. Each Transformer layer contains a self-attention mechanism and a feedforward neural network to capture semantic information at different levels. The feature aggregation unit uses the final hidden state of the CLS tag as the aggregated representation of the entire input sequence to extract global generation intent information from the complex feature matrix; The output connection unit outputs the refined deep features to a multi-class classifier for final length and class prediction. The intermediate activation state is obtained by intercepting the output tensor of a preset intermediate layer during the forward propagation of a large model.

8. The output length prediction system based on large model activation and sampling parameters as described in claim 4, characterized in that, The multi-class classification submodule is connected to the deep semantic understanding and feature refinement module. It is used to map deep features to specific length class predictions. Its network structure consists of multiple linear layers and ReLU activation functions connected in sequence. It maps the hidden state to the output length class probability distribution. The output length is divided into 5 mutually exclusive class intervals, and the predicted value is determined by the principle of maximum probability.

Images