A large model customization report generation system and method

Abstract

The application relates to the technical field of natural language processing, and discloses a large model customization report generation system and method, which comprises the following steps: a long text understanding enhancement module performs shift sparse attention calculation and low rank adaptive fine-tuning on large-scale reference literature, and outputs a fine-tuned language model; a key sentence recall module extracts a structured clue, calculates a feature vector similarity, recalls sentences, and reorganizes target context data; a controllable text generation module converts discrete constraint conditions into a control instruction sequence, combines the context data, and generates a reference standard draft through a decoding algorithm containing a length penalty factor; and a model iteration correction module calculates text gradient data according to feedback opinions, calculates candidate prompt word scores in combination with semantic alignment and distance penalty items, and regenerates a report by screening optimal prompt words. The application expands the long text processing capacity of the model, ensures that the output content strictly conforms to multidimensional specification constraints, and realizes feedback-based automatic optimization of prompt words.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a system and method for generating customized reports for large models. Background Technology

[0002] With the development of natural language processing technology, large language models have been widely applied to various text generation tasks. In customized report generation scenarios, it is often necessary to introduce a large number of standard references as background support. However, existing large language models have significant limitations in the length of input sequences when processing such ultra-long texts. Conventional global attention mechanisms lead to quadratic increases in computational resource consumption when dealing with long sequence interactions, making memory overflows highly likely when inputting long documents. To maintain model operation, input data is often truncated, which directly results in the loss of key contextual semantic information in long documents, reducing the model's accuracy in understanding the preceding input material.

[0003] Customized reports typically adhere to strict formatting guidelines, imposing explicit constraints on vocabulary, syntax, and overall length. Existing model decoding mechanisms, during autoregressive generation, often tend to output highly probable generalized words or short sentences, making it difficult to effectively integrate external discrete constraints during text prediction. This generation tendency results in draft reports with poor compliance, failing to fully output mandatory long-sentence clauses that meet regulatory requirements.

[0004] In actual business processes, the generated preliminary report draft must be reviewed and revised by domain experts. Current technologies, when optimizing the generated results using human feedback, primarily rely on business personnel manually debugging and modifying the prompts in the input model repeatedly. This debugging process is highly dependent on human experience and lacks automated evaluation methods to convert human expert feedback text into text gradients or alignment scores that the model can compute. Because the feedback intent cannot be quantified and the scope of prompt modification cannot be automatically controlled, the iterative process of prompts lacks direction, making it difficult for the system to consistently absorb expert corrections and accurately generate the report. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a system and method for generating customized reports for large models. It solves the problems that existing large language models have when generating customized reports, such as the easy loss of semantics in long text processing, difficulty in generating text that meets multi-dimensional normative constraints, and the lack of an automatic iterative optimization mechanism for prompt words based on human feedback.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The first aspect of this invention provides a large-scale customized report generation system, including a long text understanding enhancement module, a key statement recall module, a controllable text generation module, and a model iteration correction module.

[0007] Among them, the long text understanding enhancement module is used to perform shift sparse attention calculation and low-rank adaptive fine-tuning on the large-scale standard reference input data, and output the fine-tuned language model.

[0008] This module maps the input data into high-dimensional continuous long text sequence features and divides them into continuous text groups along the sequence length dimension. It also obtains the total number of attention heads in the basic pre-trained language model for logical division.

[0009] For the first half of the attention head, local self-attention calculation is performed independently within each text group to generate local attention output features; for the second half of the attention head, the text groups are shifted to the right by a preset step size in the word dimension, and the shifted feature extraction is performed based on the shifted sequence features. The output feature matrix is ​​then shifted to the left in the opposite direction to generate the reverse shifted feature matrix.

[0010] This module concatenates the local attention output features with the inverse translation feature matrix along the sequence length dimension to generate a global aggregated output feature, freezes the pre-trained weight matrix of a certain hidden layer, and introduces a low-rank update branch that cascades dimensionality reduction and dimensionality increase. The low-rank hyperparameter introduced by the dimensionality reduction matrix is ​​limited to a value range of 8 to 64 and linear superposition is performed. At the same time, the global word embedding layer and the global normalization layer are simultaneously set to a trainable state. The next word prediction task and the cross-entropy loss function are used to update the parameters, realizing feature interaction and weight update of long text sequence features.

[0011] The key statement recall module is used to extract structured clues from large-scale standard reference input data for statement recall. It uses a fine-tuned language model to retrieve and generate a set of recalled statements, and reassembles the set of recalled statements based on the received replacement and modification data instructions to generate target context data.

[0012] This module transforms the idiomatic phrases and core sentence patterns in the structured clues into structured query features, calls the fine-tuned language model to extract the state of the last hidden layer as the basic feature representation, maps the feature word sequence to query feature vectors, and maps each candidate natural language sentence to candidate feature vectors.

[0013] Solve the dot product of the query feature vector and the candidate feature vector, and divide the result of the dot product by the product of their L2 norms. At the same time, implicitly add a numerical stability constant to the denominator of the division operation to obtain a continuous scalar, to prevent numerical overflow caused by the denominator approaching zero during the division operation.

[0014] This module extracts candidate sentences based on a preset similarity threshold, which is limited to a range of 0.75 to 0.85. After secondary regular expression matching and sorting based on the metadata tags of the document tree hierarchy, the module performs fragment updates and reorganization according to manual replacement and modification data instructions, maintaining the logical sequence of text paragraphs.

[0015] The controllable text generation module is used to convert the acquired discrete constraints into natural language instructions and concatenate them into a global control instruction sequence. It combines instruction example data and target context data to generate a target prompt word sequence and outputs a reference standard draft.

[0016] This module uses a slot-filling method to instantiate discrete constraint parameters such as vocabulary, syntax, semantics, style, and length into independent instruction sentences, and introduces natural language logical connectors to perform concatenated concatenation. The built-in word segmenter extracts the target prompt word sequence into one-dimensional word index feature vectors, which, after word embedding and positional encoding, are input into a multi-layer decoder structure to call the beam search algorithm for decoding.

[0017] The final evaluation score of each candidate sequence is obtained by calculating the sum of the logarithms of the conditional probability distributions of the current word unit based on the historical word unit sequence and the target prompt word sequence in the current state, and dividing the sum of the logarithms by the maximum value between the preset length penalty factor raised to the power of the total number of word units generated by the candidate sequence and the constant 1. The value of the length penalty factor is limited to between 0.6 and 0.7 to restrict the tendency to generate short sentences and output long sentence clauses that meet the constraints.

[0018] The model iteration and correction module is used to calculate text gradient data based on the feedback data obtained for the draft reference standard, generate candidate prompt words in the opposite direction of the text gradient data to construct a candidate set, calculate the alignment score between the candidate set and the correction intention in the feedback data to select the optimal prompt word sequence, and regenerate customized report data.

[0019] This module acquires historical manual review logs and extracts structured triples containing the original target prompt word sequence, the erroneous draft reference, and expert modification feedback as historical triple data. This data is used to train an independent evaluation model. During the inference phase, the target prompt word sequence, the draft reference standard, and the feedback data are input into the independent evaluation model, aligned according to the physical paragraph index, and plain text concatenation is performed.

[0020] After multi-source concatenated text is segmented and vector-mapped, it is input into the self-attention network and feedforward neural network inside the evaluation model. The natural language description text representing logical loopholes and lack of constraints is used as text gradient data by the autoregressive output of the causal language modeling task.

[0021] The module calculates the first dot product between the candidate suggestion word sentence vector and the transpose matrix of the modified intention sentence vector. Dividing this result by the maximum value between the product of the two sentence vector norms and the numerical stability constant yields the first calculation result, representing the degree of fusion with the modified intention. It then calculates the Levinstein edit distance between the candidate suggestion word and the original target suggestion word sequence, dividing this distance by the maximum value between the original sequence character length and the constant 1 to obtain the second calculation result, limiting the modification range of the candidate suggestion word in the discrete space. This module combines a preset semantic alignment weight coefficient and a preset distance penalty weight coefficient. The first calculation result is multiplied by the semantic alignment weight coefficient, and the second calculation result is subtracted from the distance penalty weight coefficient, thereby generating a composite evaluation score for the candidate suggestion word. The semantic alignment weight coefficient is limited to a range of 0.7 to 0.8, and the distance penalty weight coefficient is limited to a range of 0.2 to 0.3. The first and second sequences are then selected for the regeneration process.

[0022] A second aspect of this invention provides a method for generating customized reports for large models, applied to the aforementioned system for generating customized reports for large models, comprising the following steps: The large-scale standard reference input data is obtained and fed into the long text understanding enhancement module. The large-scale standard reference input data is subjected to shift sparse attention calculation and low-rank adaptive fine-tuning, and the fine-tuned language model is output. The key sentence recall module extracts structured clues from large-scale standard reference input data to recall sentences. The fine-tuned language model is used to retrieve and generate a recall sentence set. The recall sentence set is reorganized based on the received replacement and modification data instructions to generate target context data. The controllable text generation module converts the acquired discrete constraints into natural language instructions and concatenates them into a global control instruction sequence. The global control instruction sequence, the acquired instruction example data, and the target context data are then concatenated to generate a target prompt word sequence. Based on the target prompt word sequence, the fine-tuned language model is used to output a draft reference standard. The model iteration and correction module calculates text gradient data based on the feedback data obtained for the draft reference standard. Candidate prompt words are generated in the opposite direction of the text gradient data to construct a candidate set. The alignment score between the candidate set and the correction intention contained in the feedback data is calculated, and the optimal prompt word sequence is selected. Customized report data is regenerated based on the optimal prompt word sequence.

[0023] This invention provides a system and method for generating customized reports for large models. It offers the following advantages: 1. This invention uses a long text understanding enhancement module to logically divide the attention head of the pre-trained language model and introduces a shift sparse feature extraction mechanism. Combined with a low-rank adaptive fine-tuning method to update local parameters, this technology can extend the context processing length of the language model for large-scale standard references without significantly increasing the model's memory usage and computational overhead. It effectively avoids the loss of semantic information caused by hard truncation during long sequence feature interactions and improves the system's ability to understand long report input data.

[0024] 2. This invention instantiates and concatenates discrete constraints of multiple dimensions such as vocabulary, syntax, and length into a global control instruction sequence through a controllable text generation module. When calling the bundle search decoding algorithm, it introduces a scoring evaluation mechanism that incorporates a length penalty factor. This technical feature limits the defect of the basic language model in the autoregressive generation process, which tends to output short sentences or generalized words. It ensures that the final generated reference standard draft strictly follows the preset format specifications and length constraints, and improves the completeness of the normative mandatory clauses output in the customized report.

[0025] 3. This invention trains an independent evaluation model using structured historical review logs through a model iteration and correction module. It outputs natural language descriptions representing logical vulnerabilities as text gradient data, and scores and filters candidate prompt words by combining semantic alignment calculation terms and edit distance penalty terms. This technical feature realizes automatic optimization of prompt words based on human feedback. While limiting the modification range of candidate prompt words to prevent basic semantic deviation, it maximizes the integration of human experts' modification intentions and reduces the system's reliance on repeated manual debugging input commands during report generation. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the system architecture of the present invention; Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Please see the appendix Figure 1 The present invention provides a large model customized report generation system, which may include a long text understanding enhancement module, a key sentence recall module, a controllable text generation module, and a model iteration correction module.

[0029] The long text understanding enhancement module receives large-scale standard reference input data, performs shift-based sparse attention computation and low-rank adaptive fine-tuning, and outputs a fine-tuned language model. A base pre-trained language model provides long sequence dependency capture capabilities. The key sentence retrieval module connects to the long text understanding enhancement module, extracts structured clues from domain references and performs sentence retrieval, receives terminal modification instructions, and outputs target context data. The controllable text generation module connects to the key sentence retrieval module, receives preset discrete constraints and converts them into natural language instructions, performs autoregressive decoding based on the natural language instructions and target context data, and outputs a draft reference standard. The model iteration and correction module connects to the controllable text generation module, receives feedback and calculates text gradients, updates prompt words based on text gradients to regenerate data, and outputs customized report data.

[0030] Please see the appendix Figure 2 This invention provides a method for generating customized reports for large models, comprising the following steps: S10: Obtain large-scale standard reference input data and input it into the long text understanding enhancement module. Perform shift sparse attention calculation and low-rank adaptive fine-tuning on the large-scale standard reference input data, and output the fine-tuned language model. Specifically, a shift sparse attention mechanism is used to perform grouped local computation and translation attention computation on long text sequences to obtain sequence feature data, and the word embedding layer and normalization layer parameters of the basic pre-trained language model are updated to complete model fine-tuning.

[0031] S20: The key sentence recall module extracts structured clues from the large-scale standard reference input data to recall sentences. The fine-tuned language model is used to retrieve and generate a recall sentence set. The recall sentence set is reorganized based on the received replacement and modification data instructions to generate target context data. Specifically, common phrases and core sentence patterns are extracted from the references, a fine-tuned language model is used to perform the search, and the generated preliminary recall statement set is sent to the interactive terminal. The data is then reorganized based on the returned replacement and modification data instructions.

[0032] S30: The controllable text generation module converts the acquired discrete constraints into natural language instructions and concatenates them into a global control instruction sequence. The global control instruction sequence, the acquired instruction example data, and the target context data are concatenated to generate a target prompt word sequence. The fine-tuned language model is used to output a draft reference standard based on the target prompt word sequence. Specifically, the system obtains preset multi-dimensional discrete constraints, converts them into natural language instructions using preset mapping rules, and concatenates them using logical connectors. Then, it combines the target context data to generate a target prompt word sequence, calls the bundle search decoding algorithm of the fine-tuned language model to perform text prediction operations, and outputs a reference standard draft.

[0033] S40: The model iteration and correction module calculates text gradient data based on the feedback data obtained for the draft reference standard. Candidate prompt words are generated in the opposite direction of the text gradient data to construct a candidate set. The alignment score between the candidate set and the correction intention contained in the feedback data is calculated, and the optimal prompt word sequence is selected. Customized report data is regenerated based on the optimal prompt word sequence.

[0034] Specifically, the feedback data, target prompt word sequence, and reference standard draft are input into the large model to trigger the prompt word optimization mechanism, output text gradient data representing the defects of the prompt words, and then generate a candidate set in reverse. The candidate selection is regarded as the best arm recognition problem to select the optimal prompt word sequence, and finally the regeneration operation is performed to output a customized report.

[0035] The large-scale standard reference input data is obtained and fed into the long text understanding enhancement module. The large-scale standard reference input data is subjected to shift sparse attention calculation and low-rank adaptive fine-tuning, and the fine-tuned language model is output. The long text understanding enhancement module receives large-scale standard reference input data and expands the context-aware window of the basic pre-trained language model by improving the attention calculation method and fine-tuning strategy. Based on the general autoregressive prediction principle of large-scale language models, this embodiment reconstructs the data flow of the underlying network layers.

[0036] In this embodiment, the input data is a one-dimensional word index sequence processed by a word segmenter. The long text understanding enhancement module maps this one-dimensional word index sequence into a high-dimensional continuous feature representation. To expand the effective receptive field and control memory overhead, the system adopts a parallel mechanism combining local attention and feature shifting (i.e., shift-sparse attention mechanism). The system acquires the long text sequence features and divides them into multiple continuous text groups along the sequence length dimension. The system obtains the total number of attention heads in the basic pre-trained language model and logically divides them. For the first half of the attention heads, the system independently performs local self-attention calculations within each text group.

[0037] The model extracts the query matrix, key matrix, and value matrix corresponding to the input feature matrix. It then calculates the dot product of the query matrix and the transpose of the key matrix, divides it by a scaling factor related to the feature dimension to smooth the gradient distribution, and maps it to attention weights using a normalized exponential function (Softmax). Finally, it multiplies these weights by the value matrix to obtain the local attention output features. Through this dimensionality reduction mapping and normalization process, the model can effectively focus on key local semantic information within text groups. For the linear projection calculation in the attention mechanism, those skilled in the art can use a standard feedforward network, and its feature mapping is a well-known technique in the field, which will not be elaborated upon here.

[0038] To establish contextual relationships between adjacent text groups, the long text understanding enhancement module abandons traditional cross-global feature interaction for the latter half of the attention head, instead relying on physical translation at the word dimension to achieve information boundary crossing. The system shifts the long text sequence features to the right by a preset step size, which is set to half the length of the text group. Based on the shifted sequence features, the system re-divides the text groups and reuses the aforementioned local attention calculation logic to perform shift feature extraction. After completing the shift feature extraction, to restore the spatial alignment of the feature matrix with the original input sequence, the system shifts the output feature matrix of this part to the left by an equal step size.

[0039] Finally, the long text understanding enhancement module performs a linear concatenation operation along the sequence length dimension between the local attention output features generated by the first half of the attention heads and the reverse translation feature matrix generated by the second half of the attention heads, thereby obtaining the global aggregated output features of this multi-head attention mechanism. By performing local computation and translation computation in parallel in different attention heads, the interactive transmission of global sequence information can be achieved while maintaining a low memory usage.

[0040] The system introduces a low-rank adaptation algorithm to update model weights. The system freezes the pre-trained weight matrix of a hidden layer in the basic pre-trained language model. Addressing the ineffectiveness of fine-tuning in ordinary low-rank adaptation fine-tuning mechanisms when the rank parameter is set too high (e.g., 256), the system introduces a specific low-rank update branch composed of a cascaded dimensionality reduction matrix and an increased dimensionality matrix. The system linearly superimposes the forward propagation output of this low-rank update branch with the output of the original pre-trained weight matrix, using this as the updated forward feature of the network layer. The low-rank hyperparameter introduced by the aforementioned dimensionality reduction matrix determines the degree of freedom of feature representation during model fine-tuning. In this embodiment, the value range of this low-rank hyperparameter is set between 8 and 64. The reason for choosing this range is that when its value is less than 8, the update branch struggles to fit the complex distribution of standard domain terminology; while when its value is greater than 64, it not only risks memory overflow but also disrupts the existing general language logic in the pre-trained weight matrix.

[0041] To compensate for the perplexity bias inherent in relying solely on low-rank updates in long-context reasoning tasks, the long-text understanding enhancement module unfreezes certain non-linear computational layers in the base pre-trained language model. Unlike the traditional approach of only updating the linear attention layer, this embodiment dynamically sets the global word embedding layer and the global normalization layer to a trainable state simultaneously, thereby bridging the performance gap between low-rank adaptation and comprehensive fine-tuning. The system thus constructs a complete trainable parameter set for the fine-tuning phase, including the low-rank matrix, global word embedding layer parameters, and global normalization layer parameters.

[0042] In the self-supervised fine-tuning phase of the model, the system uses a large-scale standard reference list as training samples and employs an autoregressive next-word prediction task to construct a self-supervised training objective. Subsequent true words in the standard text sequence are used as labels, and the cross-entropy loss function is used to quantify the deviation between the model's predicted probability distribution and the true word labels. Based on this cross-entropy loss function, the system executes a backpropagation algorithm, updating the values ​​in the overall trainable parameter set through gradient descent. Specifically, the word embedding layer update operation is used to reconstruct the absolute position encoding mapping of long sequence features, and the normalization layer update operation is used to smooth the variance fluctuations of the feature sequence over a long window span. The system outputs a fine-tuned language model based on the updated weight matrix.

[0043] The key sentence recall module extracts structured clues from large-scale standard reference input data for sentence recall. The fine-tuned language model is used to retrieve and generate a recall sentence set. Based on the received replacement and modification data instructions, the recall sentence set is reorganized to generate target context data.

[0044] The system acquires domain reference data and constructs sentence recall features. In this embodiment, the system receives a preprocessed set of document paragraphs. To meet the input requirements of the underlying language model, the system pre-processes unstructured document paragraphs with text cleaning and word segmentation, converting them into a fixed-dimensional vocabulary index sequence. The underlying language model used in this embodiment is a Transformer architecture model based on a multi-layer self-attention mechanism. The system extracts frequently occurring idioms and core sentence patterns from the historical compilation records of domain experts. These idioms specifically include expert vocabulary representing the strength of business constraints, such as "appropriate to adopt," "should be adopted," and "key algorithms." The reason for selecting this specific business data as input parameters is that normative documents have extremely strong grammatical constraints, and these idioms constitute the physical business boundary for judging whether a text fragment has strong binding force.

[0045] The system transforms the aforementioned common phrases and core sentence patterns into structured query features. The system calls the fine-tuned language model, extracts the state of its last hidden layer as the basic feature representation, and maps the extracted feature word sequence to a query feature vector. Simultaneously, the system maps each candidate natural language sentence in the reference database to a candidate feature vector. The key sentence recall module calculates the dot product of the query feature vector and the candidate feature vectors, and divides it by the product of their L2 norms to obtain a continuous scalar representing the semantic similarity score. To avoid anomalies where the denominator is zero due to the feature vector magnitude approaching zero from the underlying hardware quantization error, the system implicitly adds a very small numerical stability constant to the denominator of the division operation. Through this vector angle operation mechanism, the system can accurately measure the matching degree of discrete text sequences from the perspective of a continuous vector space.

[0046] Based on the semantic matching score, a data retrieval operation is performed to generate a recall statement set. The system obtains a preset similarity judgment threshold, which in this embodiment is specifically limited to between 0.75 and 0.85. Candidate natural language statements with semantic similarity scores greater than or equal to the similarity judgment threshold are extracted into the candidate pool.

[0047] To further verify the effectiveness of the recalled data, the system performs secondary regular expression matching on the statements in the candidate pool to determine whether their text content completely contains predefined idiomatic phrases. After dual filtering using semantic vectors and text rules, the system aligns and sorts the statements that meet the criteria according to their time dimension and chapter position in the original references. The alignment logic is implemented based on the hierarchical metadata tags of the document data to ensure the logical sequence when merging statements across documents. The system then merges and generates a preliminary set of recalled statements.

[0048] S203: Receive feedback from manual review and reorganize to generate target context data. Considering that fully automated vector retrieval is prone to semantic comprehension bias in complex engineering specifications, the key statement recall module introduces a human-in-the-loop confirmation mechanism based on terminal interaction. The system sends the initially generated recall statement set to the interactive terminal, presenting it to the reviewer in logical paragraphs through the front-end interface. The interactive terminal monitors the reviewer's interaction actions in real time and returns specific replacement and data modification instructions to the system. The system parses the above interaction instructions, accurately locates the biased data fragments in the recall statement set based on the index position information carried by the instructions, and performs synchronous text replacement updates, enabling the model to better perceive human writing preferences. During the reorganization and update process, the system strictly follows the original hierarchical structure tags of the references to logically concatenate the modified statements, avoiding contextual temporal disorder or logical gaps when merging multi-source data. The system performs format standardization and deduplication compression operations on the verified statement set, outputting target context data to support subsequent autoregressive decoding tasks.

[0049] A preset similarity threshold (a real number greater than 0 and less than or equal to 1) is obtained, and candidate natural language statements with semantic similarity scores greater than or equal to this threshold are extracted into a candidate pool. The system performs secondary regular expression matching on the statements in the candidate pool to determine whether their text content completely contains predefined idiomatic phrases. Statements that meet the criteria are aligned and sorted according to their time dimension and chapter position in the original references, based on the hierarchical metadata tags of the document data, and merged to generate a preliminary set of recalled statements.

[0050] The system acquires and quantifies multi-dimensional discrete constraints. In actual engineering standard document systems, text content is not freely distributed but is constrained by strict business physical boundaries. Based on these physical causal relationships, the system reads a set of discrete constraints for the target report text from external interaction interfaces or business setting files. The system transforms these discrete business rules into quantifiable model input parameters. To meet the multi-dimensional physical boundary specifications of standard documents in practical applications, this set of discrete constraints specifically includes subordinate feature data in five dimensions: lexical constraints, syntactic constraints, semantic constraints, style constraints, and length constraints.

[0051] The five constraints are as follows: Lexical constraints include separation constraints (requiring at least one of the original word and the set of alternative words) and exclusion constraints (requiring the exclusion of specified alternative keywords); Syntactic constraints are used to generate instructions that guide the model to adopt specific sentence structures, specifically including a specified series of part-of-speech (POS) tag sequences and structured feature constraints extracted based on the syntactic tree; Semantic constraints are used to define the core business theme, sentiment polarity, and political inclination of the output text, avoiding content digression; Style constraints are used to define the stylistic color of the text, specifically including indicators of formality, politeness, bias, toxicity, and simplicity; Length constraints use the number of lexical units as the unit of measurement and adopt a fixed interval-based calculation method (e.g., setting a span interval of 5) to set the length boundary of the output sequence. The reason for selecting the above five-dimensional constraints as input parameters is that they fully cover the core verification points of standard specification documents in compliance review, avoiding output skew caused by a single constraint control.

[0052] The system utilizes pre-defined mapping rules to transform and concatenate global control instruction sequences. Since the underlying language model cannot directly parse discrete rule codes, the system needs to transform these constraints into continuous natural language feature representations. Internally, the system maintains a mapping function based on a predefined natural language template library, using slot-filling to instantiate discrete constraint parameters of each dimension into natural language instruction text. For the extracted valid discrete constraints, the system maps them into independent instruction clauses. To achieve alignment and fusion of multi-source constraint features in the text sequence, the system introduces specific natural language logical connectors to perform concatenated concatenation operations on the instruction clauses. In this embodiment, the logical connectors preferably use the English word "and" and its corresponding multilingual variant characters. Through the physical concatenation of a one-dimensional sequence, the global aggregation of multiple constraint instructions is completed, thereby generating the global control instruction sequence text. This pure text concatenation mechanism avoids complex vector space fusion operations and directly utilizes the existing semantic understanding capabilities of the underlying model to parse composite constraints.

[0053] The system constructs a target prompt word sequence and performs autoregressive decoding. To further activate the underlying language model's ability to follow instructions under specific constraints, the controllable text generation module introduces a meta-context learning paradigm based on weakly supervised data. The system extracts structurally complete instruction example data from historical validation data, i.e., a dataset paired with constraints and standard output. The number of example data (i.e., demonstration data) is fixed at 5. The system linearly concatenates the above instruction example data, the global control instruction sequence, and the target context data output by the key statement recall module along the lexical dimension of the input sequence to construct a complete target prompt word sequence.

[0054] The system extracts the built-in word segmenter from the underlying pre-trained language model, segmenting the target prompt word sequence into one-dimensional word index feature vectors. After word embedding and positional encoding, the one-dimensional word index feature vectors are mapped into a high-dimensional continuous feature matrix and input into a multi-layer Transformer decoder structure. The masked multi-head self-attention layer inside the model calculates the hidden state of the current step based on historical context features, and finally outputs a probability distribution matrix for the global vocabulary dimension through a linear mapping layer and a Softmax activation function. To achieve controllable generation, the system simply applies a readily available generation method, calling the BeamSearch decoding algorithm to perform text prediction. During this decoding process, the system uses a logarithmic probability scoring formula including a length penalty factor to evaluate the candidate sequences. The specific calculation formula for the evaluation is as follows: ; in, This represents the cumulative log probability of the candidate sequence generation. The total number of generated lexical units, This is a length penalty factor. To balance the coherence and information content of the generated text, this embodiment limits the length penalty factor. The value ranges from 0.6 to 0.7. The system outputs the final draft text of the reference standard based on this specific scoring formula.

[0055] The model iteration and correction module calculates text gradient data based on the feedback data obtained for the draft reference standard. Candidate prompt words are generated in the opposite direction of the text gradient data to construct a candidate set. The alignment score between the candidate set and the correction intention contained in the feedback data is calculated, and the optimal prompt word sequence is selected. Customized report data is regenerated based on the optimal prompt word sequence.

[0056] The system acquires human feedback data on the draft reference standard. In real-world business scenarios, machine-generated specification texts often exhibit factual deviations in specific operating conditions or boundary indicators. Based on these physical causes and effects, the system uses a visual proofreading front-end to present the draft reference standard output by the controllable text generation module and records annotation data from domain experts on the interactive interface. The system extracts erroneous example fragments from the annotation data, along with their corresponding structured reasons for correction, and combines these to form the feedback data.

[0057] An independent evaluation model is introduced to generate text gradient data representing the defects of the prompt words. Unlike relying directly on the self-reflection of the base model, this embodiment constructs and trains an independent evaluation model. The system pre-acquires historical interaction logs and extracts historical triplet data containing the original target prompt word sequence, the erroneous draft reference, and expert modification feedback as training samples to train the evaluation model specifically. During actual inference, the system inputs the target prompt word sequence, the draft reference standard, and feedback data within the current iteration cycle into the trained independent evaluation model. The evaluation model performs logical reasoning based on multi-source context and outputs a text gradient data segment representing the logical flaws and constraint deficiencies of the current prompt word.

[0058] The system generates candidate prompts by moving in the reverse direction of the text gradient data and performs a composite scoring and filtering process. The system receives the aforementioned text gradient data, performs semantic modification operations in the reverse direction of the original target prompt sequence to generate multiple candidate prompts, and constructs a candidate set. Subsequently, the system calculates the composite evaluation score between each candidate prompt in the candidate set and the expert's correction intent. This embodiment innovatively combines semantic similarity with Levenshtein distance, and the composite scoring formula is defined as follows: ; in, Indicates candidate prompt words With the expert's intention to revise Semantic similarity between them (calculated based on the dot product of feature vectors); Indicates candidate prompt words With the original prompt words The Lewinstein edit distance between them; The semantic similarity weight is set to a value between 0.7 and 0.8. The distance penalty weight is set between 0.2 and 0.3. This composite scoring formula ensures the correct direction for prompt word repair while avoiding excessively destructive modifications to the original prompt words.

[0059] The optimal prompt word sequence is selected, and a data regeneration operation is triggered. Based on the above composite evaluation score... Based on the calculation results, the system sorts the candidate set in descending order and selects the optimal prompt word sequence with the highest composite score. This optimal prompt word sequence is then used as the input to the controllable text generation module, and the text prediction process based on the autoregressive decoding algorithm is re-executed using the fine-tuned language model. The system then presents the re-output report text to the interactive terminal and determines whether new feedback data is needed. If no new feedback instruction is received or the system's preset maximum iteration threshold is reached, the system terminates the closed-loop feedback loop and outputs the current draft reference standard as the final customized report text. Through this explicit gradient feedback fusion, the system can achieve targeted optimization of text generation quality without consuming high computational resources to fine-tune the weights within the underlying language model.

Claims

1. A large-scale customized report generation system, characterized in that, include: The long text understanding enhancement module is used to perform shift sparse attention calculation and low-rank adaptive fine-tuning on the large-scale standard reference input data, and output the fine-tuned language model. The key statement recall module is used to extract structured clues from the large-scale standard reference input data, use the fine-tuned language model to retrieve and generate a recall statement set, and reorganize the recall statement set based on the received replacement and modification data instructions to generate target context data. A controllable text generation module is used to convert the acquired discrete constraints into natural language instructions and concatenate them into a global control instruction sequence. The global control instruction sequence, the acquired instruction example data, and the target context data are concatenated to generate a target prompt word sequence. The fine-tuned language model is used to output a draft reference standard based on the target prompt word sequence. The model iterative correction module is used to calculate text gradient data based on the obtained feedback data for the draft reference standard, generate candidate prompt words in the opposite direction of the text gradient data to construct a candidate set, calculate the alignment score between the candidate set and the correction intention contained in the feedback data and filter the optimal prompt word sequence, and regenerate customized report data based on the optimal prompt word sequence.

2. The system according to claim 1, characterized in that, The long text understanding enhancement module obtains the one-dimensional word index sequence of the large-scale standard reference input data after it has been processed by a preset word segmenter, and maps it into a high-dimensional continuous feature representation. The high-dimensional continuous feature representation is divided into multiple continuous text groups in the sequence length dimension, and the total number of attention heads in the basic pre-trained language model is obtained for logical division. For the first half of the attention head that is divided, local self-attention calculation is performed independently within each text group to generate local attention output features; For the second half of the attention focus, shift the word element dimension to the right by a preset step size. Based on the shifted sequence features, re-divide the text group and perform shift feature extraction. Shift the output feature matrix obtained by shift feature extraction to the left by the preset step size to generate a reverse shift feature matrix. The local attention output features and the reverse translation feature matrix are linearly concatenated along the sequence length dimension to generate global aggregated output features. The global aggregated output features are then used for forward propagation of the network layer to output the fine-tuned language model.

3. The system according to claim 1, characterized in that, The long text understanding enhancement module obtains a basic pre-trained language model, freezes the pre-trained weight matrix of a certain hidden layer in the basic pre-trained language model, introduces a low-rank update branch composed of a cascaded dimensionality reduction matrix and an increased dimensionality matrix, wherein the value range of the low-rank hyperparameter introduced by the dimensionality reduction matrix is ​​limited to between 8 and 64, and performs linear superposition of the forward propagation output of the low-rank update branch with the output of the pre-trained weight matrix. The global word embedding layer and the global normalization layer in the basic pre-trained language model are simultaneously set to a trainable state, and an overall trainable parameter set including the low-rank matrix, the parameters of the global word embedding layer and the global normalization layer is constructed. A self-supervised training objective is constructed using the next lexical prediction task. The cross-entropy loss function is used to execute the backpropagation algorithm to update the values ​​in the overall trainable parameter set. Based on the updated trainable parameter set, the fine-tuned language model is output.

4. The system according to claim 1, characterized in that, The key sentence retrieval module extracts common phrases and core sentence patterns from the large-scale standard reference input data as the structured clues, transforms the structured clues into structured query features, calls the fine-tuned language model to extract the last hidden layer state of the fine-tuned language model as the basic feature representation, maps the extracted feature word sequence into query feature vectors, and maps each candidate natural language sentence in the large-scale standard reference input data into candidate feature vectors. Solve the dot product of the query feature vector and the candidate feature vector, and divide the dot product by the product of the second norm of the query feature vector and the candidate feature vector. Implicitly add a very small numerical stability constant to the denominator of the division operation to obtain a continuous scalar representing the semantic similarity score. A preset similarity threshold is obtained, the value range of which is limited to between 0.75 and 0.

85. Candidate natural language sentences with semantic similarity scores greater than or equal to the similarity threshold are extracted into the candidate pool.

5. The system according to claim 4, characterized in that, The key statement recall module performs secondary regular expression matching on the statements in the candidate pool to determine whether the text content of the statement completely contains the common phrase. The statements that meet the conditions are aligned and sorted according to the time dimension and chapter position in the original large-scale standard reference input data, based on the tree-like hierarchical metadata tags of the document data, and merged to generate the preliminary recall statement set. Based on the index location information carried by the obtained replacement and modification data instructions, the deviation data fragment in the recall statement set is located, and synchronous text replacement update is performed to generate the target context data.

6. The system according to claim 1, characterized in that, The discrete constraints include multi-dimensional lexical constraints, syntactic constraints, semantic constraints, style constraints, and length constraints. The controllable text generation module is based on a predefined natural language template library. It uses a slot-filling method to instantiate the parameters of each dimension in the discrete constraints into natural language instruction text and map them into independent instruction sentences. A pre-defined natural language logical connector is introduced to perform a concatenated splicing operation on the independent instruction clauses to generate a global control instruction sequence; The word segmenter built into the fine-tuned language model extracts the target prompt word sequence into a one-dimensional word index feature vector. After word embedding and position encoding, the vector is mapped into a high-dimensional continuous feature matrix. This matrix is ​​then input into the multi-layer Transformer decoder structure built into the fine-tuned language model to call the beam search decoding algorithm to perform text prediction and generate a reference standard draft.

7. The system according to claim 6, characterized in that, When the controllable text generation module calls the beam search decoding algorithm, it maintains a candidate sequence set of a preset capacity, calculates the cumulative log probability of each candidate sequence in each prediction step, and performs pruning and retention operations. The sum of the logarithms of the conditional probability distributions of the current lexical unit based on the generated historical lexical unit sequence and the target prompt word sequence in the current state of the fine-tuned language model is calculated. The sum of the logarithms is then divided by the maximum value between the power of the preset length penalty factor (the total number of lexical units generated by the candidate sequence) and the constant 1 to obtain the final evaluation score of each candidate sequence. The value of the length penalty factor is limited to between 0.6 and 0.

7. Based on the final evaluation score, the candidate sequence corresponding to the highest score is selected as the reference standard draft.

8. The system according to claim 1, characterized in that, The model iteration and correction module obtains historical manual review logs, extracts structured triples containing the original target prompt word sequence, error reference draft, and expert modification feedback as historical triple data and divides them into small batches of training data. It uses the defect direction text summarized by human experts as target labels and uses the cross-entropy loss function to execute the backpropagation algorithm to update the hidden layer weight parameters of the pre-built independent evaluation model. The target prompt word sequence, the reference standard draft, and the feedback data within the current iteration cycle are input into the independent evaluation model. Spatial alignment is performed on the physical paragraph index of the error example fragment extracted from the feedback data within the reference standard draft. A plain text concatenation operation is then performed in the sequence length dimension to generate the concatenated multi-source text. The evaluation model's built-in word segmenter is used to segment the concatenated multi-source text into a one-dimensional word feature sequence, which is then input into the word embedding layer and positional encoding layer of the evaluation model for high-dimensional vector mapping. The multi-layer masked self-attention network and feedforward neural network within the evaluation model are used to calculate the hidden state features of the concatenated sequence. An autoregressive prediction operation is performed in the vocabulary dimension using a causal language modeling task, outputting a natural language description of the constraints representing logical flaws and missing conditions as the text gradient data.

9. The system according to claim 1, characterized in that, The model iteration correction module generates candidate prompt words in the opposite direction of the text gradient data to construct the candidate set. It calculates the dot product between the sentence vector of each candidate prompt word in the candidate set after feature extraction by a preset text encoder and the transpose of the sentence vector of the correction intention contained in the feedback data after feature extraction by the text encoder. The dot product result is divided by the denominator to obtain the first calculation result. The denominator is selected as the maximum value between the product of the L2 norm of the sentence vector of the candidate prompt word and the L2 norm of the sentence vector of the correction intention and the numerical stability constant. Calculate the Levenstein edit distance between the candidate prompt word and the original target prompt word sequence, and divide the Levenstein edit distance by the maximum value between the character length of the original target prompt word sequence and a constant 1 to obtain the second calculation result; Combining a preset semantic alignment weight coefficient and a preset distance penalty weight coefficient, the first calculation result is multiplied by the semantic alignment weight coefficient, and the second calculation result multiplied by the distance penalty weight coefficient is subtracted to generate a composite evaluation score for the candidate prompt words. The semantic alignment weight coefficient is limited to a value between 0.7 and 0.8, and the distance penalty weight coefficient is limited to a value between 0.2 and 0.

3. The data with the highest ranking is extracted in descending order as the optimal prompt word sequence, and customized report data is regenerated based on the optimal prompt word sequence.

10. A method for generating customized reports for large models, characterized in that, The system for generating customized reports for large models, as described in any one of claims 1-9, comprises the following steps: A large-scale standard reference input data is obtained and input into the long text understanding enhancement module. The large-scale standard reference input data is subjected to shift sparse attention calculation and low-rank adaptive fine-tuning, and the fine-tuned language model is output. The key sentence retrieval module extracts structured clues from the large-scale standard reference input data to retrieve sentences, uses the fine-tuned language model to retrieve and generate a set of retrieved sentences, and reassembles the set of retrieved sentences based on the received replacement and modification data instructions to generate target context data. The controllable text generation module converts the acquired discrete constraints into natural language instructions and concatenates them into a global control instruction sequence. The global control instruction sequence, the acquired instruction example data, and the target context data are then concatenated to generate a target prompt word sequence. Based on the target prompt word sequence, the fine-tuned language model is used to output a draft reference standard. The model iteration and correction module calculates text gradient data based on the feedback data obtained for the draft reference standard. Candidate prompt words are generated in the opposite direction of the text gradient data to construct a candidate set. The alignment score between the candidate set and the correction intention contained in the feedback data is calculated, and the optimal prompt word sequence is selected. Customized report data is regenerated based on the optimal prompt word sequence.

Images