A self-verification decoding acceleration method and system based on document structure prior

Abstract

The application discloses a self-verification decoding acceleration method and system based on document structure prior, and relates to the technical field of artificial intelligence and natural language processing. The method obtains structure type information of a text segment to be generated to determine whether the text segment belongs to a high-frequency template area or a core data area, and calculates a structure prior score according to the structure type information; candidate token generation length and a confidence penalty coefficient are determined according to the structure prior score, a candidate token sequence is generated based on intermediate features in a shallow network of a main model, and the confidence penalty coefficient is used to correct an acceptance probability, and then complete forward propagation of the main model is performed for parallel verification. Unlike the existing scheme which depends on passive adjustment of a posterior statistical historical acceptance rate, the structure prior score can be determined in advance based on template matching degree, core data feature density and structure position prior before candidate token generation, so that adaptation delay caused by statistical lag is avoided, and the acceleration strength and verification strictness of matching can be applied to the first token of text segment type switching. The application does not need to train an independent draft model, improves the efficiency of long text generation in a vertical field, effectively guarantees the accuracy of professional terms and engineering data, and is suitable for localized deployment scenarios under restricted hardware conditions.

Description

[0001] A self-verifying decoding acceleration method and system based on document structure priors Technical Field

[0002] This invention relates to the fields of artificial intelligence and natural language processing technology, specifically to a method and system for accelerating the decoding of large language models in long text generation scenarios in vertical fields such as engineering tenders and feasibility study reports. Background Technology

[0003] With the increasing application of Large Language Models (LLMs) in enterprise-level scenarios, automatically generating specialized long texts (typically exceeding 20,000 words) such as engineering tenders and feasibility study reports has become an important means to improve enterprise content production efficiency. However, existing decoding acceleration technologies generally suffer from the following technical problems in actual deployment and application.

[0004] First, inference latency is high, and existing speculative decoding solutions rely on additional draft models. Current mainstream large language models generally employ an autoregressive word-by-word generation method, requiring a complete forward propagation computation for each token generated, leading to significant response latency when generating texts exceeding 20,000 characters. Existing speculative decoding solutions typically require training and maintaining an independent, low-parameter draft model to accelerate the main model, increasing system deployment and maintenance complexity. Furthermore, the discrepancy between the draft model and the main model in the distribution of specialized vocabulary can easily lead to lower validation pass rates when generating specialized terms in vertical fields such as bridge engineering and materials science, thus limiting the acceleration effect.

[0005] Secondly, existing adaptive speculative decoding schemes suffer from response lag and fail to fully utilize prior information about document structure. To improve the adaptability of speculative decoding across different text fragments, existing technologies have introduced schemes that dynamically adjust the speculative step size based on recent historical acceptance rates. This involves calculating the moving average acceptance rate by statistically analyzing the actual token acceptance results of the most recent several speculative steps, and then increasing or decreasing the candidate generation length for the next speculative step accordingly. However, this type of scheme is essentially a passive response mechanism based on ex-post statistics: its adjustment is entirely based on the lag statistics of already occurred verification results. When the text content switches from high-frequency template fragments (such as standard clauses and numbered lists) to core data fragments dense with technical terms and engineering data, the sliding window statistics need several speculative steps to "sense" the change in acceptance rate and make adjustments. During this lag period, the system still performs generation according to the old speculative step size that does not match the characteristics of the current fragment: either the step size is too large, causing a sharp drop in the verification pass rate of the core data fragments and a decrease in the acceleration effect instead of an increase, or the step size is too small, resulting in the acceleration potential of high-frequency template fragments not being fully released. At the same time, this type of solution does not make use of document structure information that can be known or inferred in advance before text generation (such as the type of each segment in the document outline, whether it hits the standard template library, whether it belongs to numerically dense segments, etc.), resulting in inherent defects such as insufficient information utilization and adjustment timing lagging behind the content switching boundary.

[0006] In summary, existing technologies urgently need a decoding acceleration scheme that can utilize prior information about the document structure to proactively determine the acceleration intensity and verification rigor before candidate token generation, rather than relying solely on post-event statistics and passive responses, in order to further improve the efficiency of long text generation while ensuring the accuracy of core professional content. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] The technical problem this invention aims to solve is to address the issues of high inference latency, reliance on independent draft models, and response lag and failure to utilize prior information about document structure in existing long text decoding acceleration technologies for vertical domains. This invention provides a self-verification decoding acceleration method and system based on prior information about document structure. This allows the system to proactively determine the speculation step size and verification strictness based on prior information about document structure before candidate token generation, thereby achieving a better balance between generation efficiency and accuracy of professional content without the need to train independent draft models.

[0009] (II) Technical Solution

[0010] A self-verification decoding acceleration method based on document structure prior is applied to long text generation in a vertical domain. The method includes: acquiring the structure type information of the text segment to be generated, wherein the structure type information is used to characterize whether the text segment belongs to a high-frequency template area or a core data area; calculating the structure prior score corresponding to the text segment based on the structure type information; determining the generation length of the candidate token sequence based on the structure prior score, and generating the candidate token sequence according to the generation length based on the intermediate features output by the shallow network of the main model; determining the confidence penalty coefficient based on the structure prior score, and correcting the original acceptance probability of the candidate tokens using the confidence penalty coefficient to obtain the corrected acceptance probability; calculating the target distribution using the complete forward propagation of the main model, performing parallel verification of the candidate token sequence according to the corrected acceptance probability, accepting candidate tokens that meet the conditions, and resampling rejected candidate tokens according to a preset residual distribution.

[0011] Unlike schemes that rely solely on historical acceptance rate sliding statistics for ex-post adjustments, the structural prior score described in this invention can be pre-determined based on document structure information before candidate tokens are generated, avoiding adaptation delays caused by statistical window lag. Especially at the boundary where text fragment types switch, this invention can adopt a speculative step size and verification strictness that match the characteristics of the fragment from the first token to be generated for that fragment, without waiting for a statistical observation period of several speculative steps.

[0012] Further technical details are as follows:

[0013] 1) Calculation of structural prior scores. The structural prior scores ρ are calculated by fusion as follows: ,in To determine the template matching degree between the fragment to be generated and the preset template library, The density of core data features such as numerical values, units, and technical terms in the context of this segment. The structural position prior is obtained by pre-marking the chapter type to which the segment belongs in the document outline. w1, w2, and w3 are preset weight coefficients and the sum of the three is 1.

[0014] 2) Regarding the determination of the speculative step size. The system follows... Determine the generation length of the candidate token sequence, where These are the preset lower and upper limits for candidate generation length, respectively; the higher the structural prior score (the closer to the high-frequency sample area), the larger the speculative step size and the higher the acceleration intensity.

[0015] 3) Regarding the confidence penalty coefficient and the adjusted acceptance probability. The system follows... Determine the confidence penalty coefficient, where η is the preset penalty strength coefficient; the corrected acceptance probability of the candidate token is determined according to... Calculation, where The target distribution output by the main model. This is a draft distribution of the lightweight adaptive output head based on intermediate features from the shallow layers of the main model's network. When it approaches 1 (high-frequency sample area), Approaching 1, the verification standard is basically consistent with that of conventional speculative decoding; when When it approaches 0 (core data area), A value significantly less than 1 indicates a tightened validation criterion, forcing candidate tokens to be closer to the target distribution for acceptance. This proactively strengthens accuracy assurance in areas with dense technical terminology and engineering data. For rejected candidate positions, the normalized residual distribution is applied. Resample and use the sampled data as the final output for that location.

[0016] (III) Beneficial Effects

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. No need to train and maintain independent draft models; intermediate features of the main model's own shallow network are reused to generate candidate sequences, resulting in low deployment and maintenance complexity.

[0019] 2. By using document structure prior (a fusion score of template matching degree, core data feature density and structural position prior), the text fragment type can be predicted before candidate token generation. Compared with the historical acceptance rate adjustment scheme that only relies on post-event statistics, it avoids the adaptation delay caused by statistical window lag and can apply the speculative step size and verification strategy for matching from the first token to be generated when the fragment type is switched.

[0020] 3. A confidence penalty coefficient is introduced to dynamically adjust the acceptance probability. The verification standards are actively tightened in core data segments to ensure the accuracy of professional terminology and engineering data. At the same time, the acceleration potential is fully released in high-frequency sample segments, achieving a better dynamic balance between generation speed and accuracy of professional content. This is especially suitable for single-machine local deployment scenarios with limited hardware conditions such as 16GB of video memory. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall architecture of the system described in this invention;

[0022] Figure 2 This is a flowchart illustrating the self-verification decoding acceleration method based on structural priors described in this invention. Detailed Implementation

[0023] The technical solution of the present invention will be further described below with reference to the embodiments, but the scope of protection of the present invention is not limited thereto.

[0024] I. System Overall Architecture (corresponding to) Figure 1 )

[0025] like Figure 1 As shown, the system of this invention includes a structure prior scoring module, a candidate sequence generation module, a confidence penalty verification module, and a main model complete forward verification module. The structure prior scoring module is used to obtain the structure type information of the text fragment to be generated and calculate the structure prior score; the candidate sequence generation module is used to determine the generation length based on the structure prior score and generate candidate token sequences based on the intermediate features output by the shallow network of the main model; the confidence penalty verification module is used to determine the confidence penalty coefficient based on the structure prior score and correct the acceptance probability of the candidate tokens; the main model complete forward verification module is used to perform parallel probability verification on the candidate token sequences and output the final token sequence.

[0026] II. Example 1: Self-verification decoding acceleration method based on structural prior (corresponding to) Figure 2 )

[0027] In a specific embodiment of generating the main body of the engineering tender document, let the generated historical token sequence be... The target conditional probability distribution of the main model at position t is denoted as... The system incorporates a lightweight adaptive output header at the k-th layer of the main model (k is less than the total number of layers in the model) to generate a draft distribution based on the hidden state at the k-th layer, denoted as q(x). t |x_{ <t})。

[0028] For the current position t to be generated, the system first extracts the structural prior information of the text segment at that position, including: the template matching degree obtained by matching the segment with a pre-set template library. The core data feature density s_data(t) obtained by numerical, unit, and technical terminology detection of the fragment context; and the prior structural location obtained by pre-annotation based on the document outline. The three factors are combined according to preset weights to obtain the structural prior score:

[0029]

[0030] The system is based on ρ t Determine the speculative step size for this round, which is the number of candidate tokens generated at once by the lightweight output head:

[0031]

[0032] At the same time, according to ρt Determine the confidence penalty coefficient for this round:

[0033] η is the preset penalty intensity coefficient.

[0034] The lightweight output head generates γ based on the k-th hidden state. t A draft sequence consisting of candidate tokens is generated; then the main model performs a complete forward propagation on the draft sequence, and computes the target distribution at each candidate position in parallel. The modified acceptance probability of a candidate token is calculated using the following formula:

[0035]

[0036] For the first rejected position, a token is resampled according to the normalized residual distribution of the target distribution and the draft distribution as the final output for that position, and subsequent candidate tokens are discarded. The residual distribution is defined as follows:

[0037]

[0038] Where norm(·) represents the normalization process. Since ρ t The speculative step size γ described in this embodiment is determined based on the document structure information before candidate token generation. t With confidence penalty coefficient κ(ρ) t All of these are "proactively set in advance" rather than "passively adjusted statistically afterward." In high-frequency template segments such as standard clauses and numbered lists, ρ t The system is relatively high, increasing the speculative step size to improve generation throughput, while the confidence penalty is close to zero; in core data segments with dense technical terms and engineering data, such as CFRP reinforcement materials and bridge span data, ρ t The system is relatively low. On the one hand, it shrinks the speculative step size to reduce the cost of single-step verification failure. On the other hand, it actively tightens the acceptance probability through the confidence penalty coefficient, so that the candidate token must be closer to the target distribution of the main model in order to pass the verification. Thus, the matching and adjustment of acceleration intensity and verification strictness are completed from the first token of the segment switching.

[0039] III. Methods for Obtaining Prior Structural Information

[0040] In a specific embodiment, the template matching degree s_tpl(t) is obtained as follows: A template library is constructed by pre-collecting frequently repeated texts such as standard clauses, fixed expressions, chapter titles, and numbered lists from historical enterprise tenders and feasibility study reports; the n-gram overlap or edit distance is calculated between the fragment to be generated and the entries in the template library, and the similarity corresponding to the entry with the highest matching degree is taken as s_tpl(t). The core data feature density s_data(t) is obtained as follows: Regular expression matching (numerical values, units, dates, specification numbers, etc.) and named entity recognition (technical terms, material grades, etc.) are performed on the context of the fragment to be generated (such as already generated outline section descriptions and reference fragments); the frequency of occurrence of core data features within a unit length of text is statistically analyzed and normalized to obtain s_data(t). The structural position prior s_pos(t) is obtained as follows: During the document outline generation stage, the type of each node in the outline is pre-labeled (such as standard clause type, data table type, explanatory text type, etc.); during the generation process, the corresponding position prior value is read based on the current outline node. All three types of information mentioned above can be obtained or calculated in advance before candidate token generation, thereby ensuring the structural prior score ρ. t The "pre-emptive" nature of it.

[0041] IV. Specific Implementation Environment

[0042] The method described in this invention can be deployed on a local Ubuntu or WSL2 runtime environment, supporting single-machine hardware configurations with 16GB of video memory. The main model can adopt a base model with 7B parameters. In actual engineering deployment, the decoding acceleration method described in this invention can be used as a core component of the inference engine, deployed in conjunction with peripheral service components such as upper-layer request access, identity authentication, and task scheduling to support unified access management in multi-task and multi-user scenarios. The aforementioned peripheral scheduling components are used to facilitate the engineering implementation of the method described in this invention and do not constitute essential technical features of this invention. Furthermore, considering the security of core data such as enterprise engineering cost data and professional technical materials, the inference engine described in this invention does not rely on any external remote service calls during operation and actively blocks and isolates third-party remote tracking and telemetry components (e.g., explicitly removing data backhaul links from large model observability platforms such as LangSmith), ensuring that the collection, processing, and storage of core data are entirely confined within the local physical boundary, achieving absolute physical isolation between enterprise core data and the external network environment under limited hardware resources.

[0043] It should be understood that the above embodiments are merely preferred embodiments of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Any modifications, equivalent substitutions, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A self-verifying decoding acceleration method based on document structure priors, applied to long text generation in vertical domains, characterized in that... Includes the following steps: Obtain the structural type information of the text fragment to be generated, wherein the structural type information is used to characterize whether the text fragment belongs to the high-frequency template area or the core data area; Calculate the prior structure score corresponding to the text fragment based on the structure type information; The generation length of the candidate token sequence is determined based on the prior score of the structure. Based on the intermediate features output by the shallow network of the main model, the candidate token sequence is generated according to the generated length to obtain multiple candidate tokens under the draft distribution; Based on the prior scores of the structure, the confidence penalty coefficient is determined; the original acceptance probability of the candidate token is corrected using the confidence penalty coefficient to obtain the corrected acceptance probability; The target distribution is calculated using the complete forward propagation of the main model. The candidate token sequence is then verified in parallel according to the modified acceptance probability. Candidate tokens that meet the modified acceptance probability condition are accepted. The rejected candidate tokens are resampled according to the preset residual distribution to obtain the final output token.

2. The method according to claim 1, characterized in that, The structural prior score is calculated as follows: Where, ρ t Let be the structural prior score at position t. The degree of matching between the text fragment and the templates in the preset template library. The core data feature density of numerical values, units, and technical terms in the context of the text fragment. The structural position prior is obtained by pre-labeling the chapter type of the text fragments described in the document outline. w1, w2, and w3 are preset weight coefficients and w1+w2+w3=1.

3. The method according to claim 2, characterized in that, The step of determining the generation length of the candidate token sequence based on the prior score of the structure includes: determining the generation length according to the following formula. in, , These are the preset lower and upper limits for candidate generation length, respectively.

4. The method according to claim 2 or 3, characterized in that, The confidence penalty coefficient is calculated as follows: Where η is a preset penalty intensity coefficient; the corrected acceptance probability is calculated as follows: in, The target distribution output by the main model. The draft distribution is output by the lightweight adaptive output head based on the intermediate features.

5. The method according to claim 4, characterized in that, The preset residual distribution is calculated as follows: Where norm(·) represents normalization.

6. The method according to claim 1, characterized in that, The process of obtaining the structural type information of the text fragment to be generated includes at least one of the following methods: performing n-gram matching or edit distance matching between the text fragment and a pre-set template library to obtain the template matching degree; performing regular expression matching or named entity recognition on the context of the text fragment to obtain the core data feature density; Based on the type labels pre-marked on each chapter of the document according to the document outline, the prior structural position is obtained.

7. A self-verifying decoding acceleration system based on prior knowledge of document structure, characterized in that, include: The structure prior scoring module is used to obtain the structure type information of the text fragment to be generated, and to calculate the structure prior score based on the structure type information. The candidate sequence generation module is used to determine the generation length of the candidate token sequence based on the structural prior score, and generate the candidate token sequence according to the generation length based on the intermediate features output by the shallow network of the main model; the confidence penalty verification module is used to determine the confidence penalty coefficient based on the structural prior score, and use the confidence penalty coefficient to correct the original acceptance probability of the candidate token; the main model verification module is used to calculate the target distribution using the complete forward propagation of the main model, and perform parallel verification of the candidate token sequence according to the corrected acceptance probability.

8. The system according to claim 7, characterized in that, The structure prior scoring module is specifically used to calculate the structure prior score in accordance with the manner described in claim 2.

9. The system according to claim 7, characterized in that, The confidence penalty verification module is specifically used to calculate the confidence penalty coefficient and the corrected acceptance probability in accordance with the method described in claim 4.

10. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 6.

Images