A method for improving the quality of generated text based on a large language model

Abstract

This invention relates to the field of natural language processing technology, and discloses a method and system for improving the quality of generated text based on a large language model. The method includes: constructing a logical topology graph and Laplacian matrix of the original generated text and extracting a sequence of feature values; identifying logical load-bearing walls based on attention gradients and generating counterfactual comparison text; calculating a logic collapse index by combining the differences in spectral feature values ​​and logical polarity reversals between the original and comparison texts; determining the quality of the generated text based on the logic collapse index; blocking texts that fail the determination; and using spectral difference information to reshape and resample the output probability values ​​at erroneous locations to generate new text. This invention does not rely on an external knowledge base, quantifies the stability of text logic through counterfactual interference and spectral analysis, identifies texts with high-risk illusions, and automatically corrects erroneous logical paths through Logits reshaping, improving the logical consistency of the generated content while ensuring the semantic fluency of the text.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a method and system for improving the quality of generated text based on a large language model. Background Technology

[0002] With the rapid development of deep learning technology, Large Language Models (LLMs) based on the Transformer architecture have made groundbreaking progress in the field of natural language processing. These models have demonstrated extremely high fluency and generalization ability in tasks such as text generation, machine translation, and intelligent question answering, and are gradually being applied to professional fields with high requirements for logical rigor, such as intelligent healthcare, legal document drafting, and financial research report analysis.

[0003] However, in practical applications, especially when dealing with factual texts with rigorous logical structures such as medical and legal texts, traditional large language models have some shortcomings:

[0004] First, the generation mechanism of existing large language models is actually based on probabilistic sampling to predict the next token. This generation mechanism tends to prioritize high-probability tokens that conform to language habits, lacking verification of the deep logical topology between entities. Therefore, although it can ensure the fluency and grammatical correctness of the generated text, it is prone to producing the illusion of fluent language but illogical logic when dealing with factual texts with high logical density, strict causal chains, and reliance on entity relationships. Specifically, the text generated by the model may read smoothly, but its internal logic may be contradictory, causally reversed, or even fabricated. This problem of fluent but illogical generated content restricts the application of large language models in scenarios with high reliability requirements.

[0005] Secondly, existing methods for quality detection and optimization of large language models primarily rely on retrieval-enhanced generation techniques. These methods introduce external knowledge bases and retrieve relevant facts during the generation process for calibration. However, this reliance on external knowledge often proves ineffective in situations involving niche domains lacking external knowledge base coverage, private data scenarios, or outdated knowledge bases. In such cases, the system cannot effectively judge the logical consistency of the generated content based solely on the model's capabilities, leading to a failure to promptly detect and correct potential logical fallacies. Therefore, a method and system for processing generated text quality based on large language models is urgently needed to address these issues. Summary of the Invention

[0006] In view of the problems in related technologies, the present invention provides a method for generating text quality processing based on a large language model to overcome the technical problems existing in the prior art.

[0007] To solve the aforementioned technical problem, the present invention is achieved through the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide a method for quality processing of generated text based on a large language model, specifically including: generating original text based on prompts input by the user, constructing a logical topology graph and a Laplace matrix, performing eigenvalue decomposition on the Laplace matrix to obtain the eigenvalue sequence of the original graph; calculating the load-bearing coefficient of each entity in the original text based on attention gradients, analyzing and identifying logical load-bearing walls and implementing counterfactual semantic flipping to guide the large language model to generate counterfactual comparison text; constructing a comparison logic graph and Laplace matrix eigenvalue sequence of the counterfactual comparison text, calculating a logic collapse index by combining the eigenvalue differences between the original text and the counterfactual comparison text; comparing the logic collapse index with a preset quality safety threshold to determine the quality of the generated text; performing a blocking operation on text that fails the quality determination, and simultaneously performing reshaping and sampling on the output probability values ​​of the erroneous positions of the logic collapse to generate new text;

[0009] The weight coefficient of each entity in the original text is calculated based on the attention gradient. Its expression is:

[0010]

[0011] In the formula, For the number of attention heads, This is the aggregate semantic representation vector of the entire sentence text. For entities exist Feature vector representation in each attention head;

[0012] Select the load-bearing coefficient The tallest entity acts as a logical load-bearing wall;

[0013] The expression for calculating the logic collapse index is as follows:

[0014]

[0015] In the formula, The logic collapse index. To generate text based on the original Directed logic graph, For counterfactual comparison text The comparison logic diagram, For the first One eigenvalue; This is the polarity reversal penalty coefficient; This is a logical reversal flag; it takes a value of 1 if the main conclusion changes from affirmative to negative, and 0 otherwise.

[0016] The generation of new text specifically includes:

[0017] Compare with the original directed logic graph Logic diagram of contrast with counterfactual facts Identify the set of edges with the largest weight changes and polarity reversals, and map relation descriptors back to the set of erroneous word indexes in the original text;

[0018] Construct a sparse mask vector, assigning a value of 1 to the position corresponding to the erroneous word and 0 to the rest;

[0019] At the decoding stage of the large language model regeneration, the original prediction vector output by the large language model is obtained, and the original prediction vector is corrected using the logistic collapse index to obtain the corrected vector.

[0020] The modified vector is subjected to Softmax normalization to obtain a probability distribution, and new text is generated by sampling based on the probability distribution.

[0021] As a preferred embodiment of the text quality processing method based on a large language model described in this invention, the construction of the logical topology graph and Laplace matrix specifically includes: generating text from the original text using an entity relation extraction model. Extract key entity set With relation set Construct a directed logic graph ; Calculate the directed logic graph degree matrix Adjacency Matrix Construct the Laplace matrix .

[0022] As a preferred embodiment of the text quality processing method based on a large language model described in this invention, the eigenvalue sequence of the original spectrogram is obtained by solving for the roots of the characteristic polynomial, specifically including: [the following is a partial translation of the original text, which is not directly related to the preceding sentence and can be omitted:] ...on the Laplace matrix... Perform eigenvalue decomposition and solve for the characteristic polynomial. The root is used to obtain the original spectral feature value sequence arranged in ascending order. ;in, This represents determinant calculation. It is the identity matrix. Then it represents the eigenvalue.

[0023] As a preferred embodiment of the text quality processing method based on a large language model described in this invention, the step of analyzing and identifying logical load-bearing walls and implementing counterfactual semantic flipping specifically includes:

[0024] Locate the core attribute edge of the logical load-bearing wall in the logical topology graph, and use a semantic adversarial generator to generate a logically mutually exclusive description of the core attribute edge;

[0025] Construct distractor words containing counterfactual assumptions It is then injected into a large language model, forcibly guiding the model to generate counterfactual contrast text. .

[0026] As a preferred embodiment of the text quality processing method based on a large language model described in this invention, the step of determining the quality of the generated text specifically includes: setting a quality safety threshold. ;

[0027] If the logic collapses index The hallucination was determined to be of high risk, and a blocking operation was performed.

[0028] If the logic collapses index If the illusion is determined to be low-risk, the text will be output to the user.

[0029] As a preferred embodiment of the text quality processing method based on a large language model described in this invention, the quality safety threshold is... The Yoden index was used for optimization, specifically including: constructing a logistic calibration dataset containing positive and negative samples, calculating the logistic collapse index for each sample, and iterating through candidate thresholds. Calculate the Youden index ,in, The true positive rate, The false positive rate; select the value that makes the Youden index... The largest candidate threshold is used as the quality and safety threshold. .

[0030] Secondly, embodiments of the present invention provide a text quality processing system based on a large language model, comprising: a logic graph spectrum analysis module, used to generate original text based on user prompts, extract entity relationships to construct a directed logic graph and Laplacian matrix, and perform eigenvalue decomposition to obtain the original graph feature value sequence; a counterfactual interference generation module, used to identify logical load-bearing walls in the original text based on attention gradients, implement counterfactual flipping on their core attributes to construct interference prompts, and guide the large language model to generate counterfactual comparison text; a logic collapse assessment module, used to construct a comparison logic graph and feature value sequence of the counterfactual comparison text, and quantitatively calculate the logic collapse index by combining the differences in the feature value sequence of the original graph; a quality judgment module, used to compare the logic collapse index with a preset quality safety threshold, and determine the quality of the generated text based on the comparison result; and a self-correcting optimization module, used to suppress and resample the output probability values ​​of erroneous paths in the logic collapse during the model regeneration process, and generate new text.

[0031] The present invention has the following beneficial effects:

[0032] 1. This invention introduces a counterfactual interference mechanism and a logic collapse index. By identifying logical load-bearing walls in the text and applying semantic flipping to generate counterfactual contrast text, the distance between the original text and the counterfactual contrast text in terms of graph features (logic collapse index) is calculated. This mathematically quantifies whether the generated text possesses robust causal logic. This enables the system to effectively distinguish between normal rhetorical text and potential logical fallacies, helping to avoid the problem of traditional methods focusing only on linguistic fluency while neglecting logical rigor.

[0033] 2. This invention constructs a logical topology graph of the original generated text and performs spectral analysis, transforming the text's inherent logical structure into a quantifiable sequence of feature values. Compared to existing retrieval-dependent augmentation generation techniques, this invention does not rely on external encyclopedias, databases, or internet information; it can complete quality detection solely using the output features of the large language model itself. This solves the problem of being unable to determine the logical consistency of generated content due to a lack of external reference materials in scenarios involving private data, niche fields, or emerging new knowledge.

[0034] 3. For high-risk hallucinatory texts detected, this invention can not only intercept them but also perform automatic correction at the model decoding layer. By locating the specific word units that cause logical collapse and dynamically calculating the suppression strength based on the collapse index, the output probability of erroneous paths is suppressed during model regeneration. This forces the large language model to avoid known erroneous logical branches and explore other high-probability and logically consistent generation paths, thus achieving a closed loop from error detection to error correction without disrupting the overall semantic coherence of the text.

[0035] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, the drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a flowchart of a text quality processing method based on a large language model provided in Embodiment 1 of the present invention.

[0038] Figure 2 This is a schematic diagram of the module logic interaction of a text quality processing system based on a large language model, provided in Embodiment 2 of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0040] When processing factual texts with rigorous logical structures, such as those in the medical and legal fields, the correctness of the text depends on the stability of entity relationships. However, the generation mechanism of traditional Large Language Models (LLMs) is based on lexical probability sampling. This mechanism tends to prioritize high-probability lexical terms that conform to language habits, lacking verification of deep logical topologies between entities. Therefore, when faced with complex logical reasoning, traditional LLMs often prioritize linguistic fluency while neglecting the logical stability of entity relationships, easily leading to the illusion that the generated content is linguistically fluent but internally logically contradictory. Furthermore, existing quality detection methods mostly rely on external knowledge base retrieval (RAG), and in scenarios lacking external knowledge base coverage, they cannot judge the logical consistency of the generated content solely based on the model itself.

[0041] To solve the above technical problems, such as Figure 1As shown, Embodiment 1 of the present invention provides a method for processing the quality of generated text based on a large language model. Specifically, Embodiment 1 uses a high-risk intelligent medical question-and-answer scenario as an example: A hospital has deployed a private large-model system to assist doctors in conducting pharmacological consultations. Users (doctors or patients) ask the system professional questions about specific symptoms and drug contraindications. It should be noted that while Embodiment 1 uses a medical scenario as an example, the present invention is also applicable to other scenarios requiring high logical rigor, such as legal document generation and financial research report writing.

[0042] In the specific implementation of Example 1: First, the original generated text is generated based on the prompts input by the user, and a logical topology graph and Laplace matrix are constructed. Eigenvalue decomposition of the Laplace matrix yields the eigenvalue sequence of the original graph. This method transforms the logical structure of the text content into a quantifiable spectral feature sequence, establishing a mathematical benchmark for measuring the logical stability of the original text. Second, the load-bearing coefficient of each entity in the original text is calculated based on the attention gradient. Logical load-bearing walls are analyzed and identified, and counterfactual semantic flipping is implemented to guide the large language model to generate counterfactual comparison text. This method quantifies the gradient sensitivity of entities to the generated results, locates load-bearing wall entities that maintain the current logical validity, and generates strong interference samples through counterfactual flipping, effectively stimulating potential logical vulnerabilities and providing comparative material for subsequent difference assessment. Then, a comparison logic graph and Laplace matrix eigenvalue sequence of the counterfactual comparison text are constructed. Combining the eigenvalue differences and logical polarity flipping between the original text and the counterfactual comparison text, the logical collapse index is calculated. This method uses mathematical methods to quantify the logical structure stability of the original text when encountering counterfactual attacks, providing an objective mathematical basis for risk assessment. Next, the logic collapse index is compared with a quality and safety threshold to determine the quality of the generated text. If the logic collapse index is greater than or equal to the quality and safety threshold, the generated text is determined to be a high-risk illusion and a blocking operation is performed; if the logic collapse index is less than the quality and safety threshold, it is determined to be a low-risk illusion and the text is output. This method can effectively block generated content with serious logical fallacies while allowing normal rhetorical expressions, balancing the system's security and usability. Finally, a blocking operation is performed on text that fails the quality judgment. At the same time, spectral difference information is used to shape and sample the output probability values ​​of the logical collapse error locations to generate new text. This method forces the large language model to avoid known error branches without compromising sentence fluency by suppressing the probability of erroneous logical paths, achieving logical self-correction without the need for an external knowledge base.

[0043] Furthermore, to better illustrate the technical solution of Embodiment 1 of the present invention, a detailed description of the text quality processing method based on a large language model is provided, specifically including the following:

[0044] S1. Construct the logical topology graph and Laplace matrix of the original generated text, specifically including the following sub-steps:

[0045] S11. First, the system receives prompts from the user, such as "Can patients with high blood pressure take ibuprofen?". The large model performs initial reasoning and generates the original text. "People with high blood pressure can take ibuprofen because it relieves pain and has minimal impact on blood pressure." (Original text) The text is a suspected hallucination to be detected.

[0046] S12. The system utilizes an entity relationship extraction model to extract data from the original text. Extract key entity set With relation set Construct a directed logic graph Specifically, extract nodes. (hypertension), (ibuprofen) (Minimally affected by blood pressure) (Pain relief); Extracting edge relationships ( (Permitted to take) ( ,Function), ( (Side effects properties).

[0047] In this embodiment 1, the entity relationship extraction model extracts the key entity set. With relation set The steps include: First, the system processes the original text. Inputting the BERT pre-trained model yields a context-aware word vector sequence H. Next, the first-layer pointer network scans the sequence H, using a sigmoid activation function to predict the probability that each word belongs to the first or last character of an entity, thereby extracting all entity nodes that serve as logical subjects. Subsequently, for each extracted subject... The system fuses its feature vectors back into sequence H and inputs them into a second-layer pointer network corresponding to different logical relationship categories (such as "cause and effect", "contrast", "parallel"). This second-layer network searches for the corresponding object entity under the constraint of a specific subject. This layered structure can resolve the issues of entity overlap and nesting in text (e.g., "hypertension" and "hypertensive patients"). Finally, the system extracts the triples ( ,relation This is transformed into a directed graph structure. The relation represents the relationship between the subject and object. Specifically, in constructing the edge set... At this time, the system initializes the weights of the edges according to the semantic attributes of the relation type. For example, strong causal relations such as "therefore" and "lead to" are assigned high weights (e.g., 1.0); weak modal relations such as "maybe" and "suggest" are assigned low weights (e.g., 0.5); and adversative relations such as "but" and "however" are marked as negative connections or specific phases in the Laplace matrix, thereby preserving the logical polarity of the text at the graph level.

[0048] S13, The system is based on the directed logic diagram. Constructing the Laplace matrix The specific implementation steps include:

[0049] S131. First, construct the adjacency matrix. degree matrix :

[0050] Wherein, degree matrix It is The diagonal matrix, and The number of nodes. The degree matrix... Column elements Equal to node The degree. In this embodiment 1, for a directed graph, the sum of the out-degree and in-degree is used to measure the connection strength of a node. For example, if a node... (Ibuprofen) has one incoming edge (related to high blood pressure) and two outgoing edges (indicating pain relief and minimal impact on blood pressure), so its degree is 1+2=3, that is... .

[0051] Among them, the adjacency matrix It is The matrix, if the nodes and If there is an edge, then Otherwise, it is 0.

[0052] The formula for calculating the Laplace matrix is: This formula indicates that the diagonal of the Laplace matrix represents the degree of each node, and the off-diagonal positions (…). If there is an edge connecting them, the result is -1; otherwise, it is 0.

[0053] S132, Laplace matrix Perform eigenvalue decomposition:

[0054] Obtain the Laplace matrix Next, we need to find a set of scalars. This makes the Laplace matrix Its effect in a specific vector direction is merely scaling, specifically achieved by solving for the roots of the characteristic polynomial: ,in This represents determinant calculation. It is the identity matrix. Then it represents the eigenvalue.

[0055] Calculate all eigenvalues Then, arrange them in ascending order to obtain the feature value sequence of the original map. And satisfy .

[0056] In this embodiment 1, by calculating the eigenvalues ​​of the original directed logic graph Laplace matrix, the logical structure of the text content is transformed into a quantifiable spectral feature sequence, thereby establishing a mathematical benchmark for measuring the logical stability of the original text. Specifically, for example, regarding the aforementioned original text... and the constructed directed logic graph The calculated eigenvalue sequence is as follows: This sequence of eigenvalues ​​reflects the tightness of the logical connections between nodes (such as "ibuprofen" and "small effect on blood pressure").

[0057] S2. Identify logical load-bearing walls based on attention gradient analysis and generate counterfactual interference samples, specifically including the following sub-steps:

[0058] S21. To avoid invalid random perturbations, the system calculates the original text. The load-bearing capacity of each entity in the middle The calculation expression is:

[0059]

[0060] In the formula, For the number of attention heads, This is the aggregate semantic representation vector of the entire sentence text. For entities exist The feature vector representation in each attention head.

[0061] In this embodiment 1, the load-bearing coefficient of each entity in the original text is calculated. The principle is to view the generation process of a large language model as a non-linear mapping function from the input feature space to the output semantic space, and to quantify the logical contribution of entities by calculating the sensitivity of the output to the input. The specific process is as follows:

[0062] a. Define the output semantic target and input feature representation: Take the hidden state of the last token in the text sequence generated by the large language model as the aggregate semantic representation vector of the entire sentence, denoted as... For any entity in the text Internally, it is represented as a set of feature vectors. Example 1 employs a Transformer-based multi-head attention structure model architecture, where the entity... The features do not exist as a single vector, but are distributed throughout the model. Processed in parallel within independent attention heads, therefore assume For entities exist Feature vector representation in each attention head;

[0063] b. Constructing a single-head contribution based on gradient sensitivity: According to the first-order Taylor expansion principle, if the input features undergo a small perturbation, the change in output is proportional to the partial derivative (gradient). Therefore, the output semantic vector... Relative to input entity features The gradient directly reflects the entity's sensitivity to the generated result. For the th... Each attention head is used to calculate the gradient vector. The strength of the gradient vector is quantified using the Euclidean algorithm to obtain the sensitivity index under this head: , indicating when the Medium-sized solid The degree to which the final semantic result is affected when the information undergoes slight fluctuations;

[0064] c. Multi-head aggregation calculation logic load factor: In order to obtain the comprehensive evaluation index of the entity at the entire model level, it is necessary to aggregate the information of all attention heads and normalize it using the arithmetic mean method to obtain the entity's... Logical load-bearing coefficient The expression is:

[0065]

[0066] d. Perform calculations and sorting: The system iterates through all entities generated in the text. Calculate the logical load-bearing coefficient using the formula described above. .

[0067] S22, The system identifies the logical load-bearing coefficient. The tallest entity acts as the logical load-bearing wall.

[0068] S23. Implement counterfactual semantic flipping for logical load-bearing walls and construct interference prompts. The prompt word is then injected into the large language model, forcing it to regenerate the text under the assumed conditions, resulting in a counterfactual text. It should be noted that if the counterfactual text is used... The number of feature values ​​and the original text If the number of feature values ​​is inconsistent, the zero-padding method is used to pad them to the same dimension.

[0069] In this embodiment 1, counterfactual semantic flipping is implemented to construct interference prompts. The specific steps are as follows: First, the system is based on a directed logic graph. The system first locates the core attribute edge (e.g., "no side effects") of the logical load-bearing wall (e.g., "ibuprofen") in the current context. Then, the system calls an existing semantic adversarial generator, which specifically employs a gradient-ascending feature perturbation network. This generator constructs an adversarial loss function for the core attribute (e.g., "no side effects"), calculates the perturbation vector that minimizes the semantic consistency probability using the Fast Gradient Sign Method (FGSM), and superimposes this perturbation vector onto the embedding vector of the original attribute word. This generates a logically mutually exclusive description of the core attribute edge based on the word vector perturbation. Specifically, the system finds the descriptive word with the minimum cosine similarity (i.e., opposite direction) to the original attribute vector in the semantic vector space, or uses preset logical non-templates (e.g., mapping "safe" to "highly toxic," and "insulating" to "conductive") to generate counterfactual attribute descriptions. Finally, the system constructs interference prompts containing strong assumptions. To ensure that the large language model can respond to this counterfactual condition first, this embodiment adopts a pre-hypothesis injection strategy, that is, constructing an instruction in the format of "If we force [entity] to have the characteristic of [counterfactual attribute], please answer [original question] under this premise".

[0070] In this embodiment 1, by quantifying the gradient sensitivity of entities to the generated results, the load-bearing wall entities that maintain the current logic are accurately located. Furthermore, strong interference samples are generated through counterfactual flipping, thereby providing effective test cases for subsequent stress testing. Specifically, for example, the number of attention heads is set. After backpropagation calculation using the model, the logical load-bearing coefficients of each entity are as follows: , , , The system then calculates the logical weighting coefficient of the relationship between the entity "ibuprofen" and "minor impact on blood pressure". At its highest level, determine the entity. (Ibuprofen) serves as the logical load-bearing wall; subsequently, the system automatically constructs counterfactual prompts: "If ibuprofen is a potent vasoconstrictor...", and then forces the large language model to generate counterfactual contrast text. "If ibuprofen is a vasoconstrictor, then patients with high blood pressure should absolutely not take it, because it will significantly raise blood pressure and cause life-threatening situations."

[0071] S3. Perform dynamic spectral analysis to calculate the logic collapse index, which includes the following sub-steps:

[0072] S31, Counterfactual Text Using the entity relationship extraction model described in step S12 above, a comparison logic diagram is constructed. At this point, the structure of the new graph changes: the original affirmative connections... It becomes a negative connection (allow → prohibit), and the newly enhanced high-weight edge of the association (leads to → life-threatening situation).

[0073] S32, Calculation comparison logic diagram Laplace matrix eigenvalue sequence In this embodiment 1, .

[0074] S33. The system uses the Wasserstein distance concept to calculate the logic collapse index. The expression for quantifying the degree of deformation of the original text under stress testing is:

[0075]

[0076] In the formula, For the first One eigenvalue; This is the polarity reversal penalty coefficient; This is a logical reversal flag. It takes the value 1 if the main conclusion changes from affirmative to negative, and 0 otherwise.

[0077] It should be noted that the polarity reversal penalty coefficient An adaptive setting method based on the maximum bandgap of the logic graph is employed. Specifically, the system obtains the eigenvalue sequence from the original logic graph. Extracting the largest eigenvalue This value represents a directed logic graph in algebraic graph theory. The system sets the hardness of the strongest logical connection, based on the physical premise that polarity reversal is equivalent to logical breakage, and sets a penalty coefficient accordingly. Equal to the largest eigenvalue: This indicates that once a logical polarity reversal occurs (i.e., The penalty imposed by the system will be equivalent to the energy required to completely destroy the core logical chain of the text, which ensures... The penalty is always directly proportional to the logical tightness of the current text; that is, the more rigorous the text logic, the heavier the penalty for polarity reversal.

[0078] In this embodiment 1, the logical collapse index is calculated to quantify the logical structural stability of the original text when subjected to counterfactual attacks, thereby providing a mathematical basis for subsequent risk assessment. Specifically, for example, the original logic diagram is obtained based on the above calculation. eigenvalue sequence and the corresponding logic diagram Laplace matrix eigenvalue sequence Calculate the logic collapse index At this point, due to the counterfactual comparison text The conclusion changes from "can be taken" (affirmative) to "cannot be taken" (negative), thus the logical polarity is reversed. Furthermore, the penalty coefficient is set according to the adaptive rules. The logic collapse index was calculated. .

[0079] S4. Determine the quality of the generated text: Set the system's quality and safety threshold. The logic collapse index With quality and safety thresholds In comparison, quality and safety safeguards are implemented, specifically including the following steps:

[0080] If the logic collapses index Explanation of the original text The logical structure collapsed catastrophically under minor changes in assumptions. That is, the large language model lacked a deep understanding of the argument it generated that "has minimal impact on blood pressure", judged it as a high-risk illusion, and executed a blocking operation.

[0081] If the logic collapses index Explanation of the original text The logical structure is correct, it is determined to be a low-risk illusion, and the text is output to the user.

[0082] In this embodiment 1, the quality and safety threshold The quality and safety thresholds were determined using an optimization method based on the ROC curve (Receiving Operating Characteristic) curve of the gold standard calibration set to balance the false alarm rate and the false negative rate. The specific steps are as follows:

[0083] S41. Construct a logical calibration dataset: Select A known sample labeled by human experts, containing A logically rigorous, high-quality text (positive sample) and A piece of illusory text containing obvious logical fallacies (negative sample);

[0084] S42. Batch Calculation of Collapse Index: Input each text in the calibration dataset into the system, calculate the logistic collapse index of each sample, and obtain the positive sample index set. With negative sample index set Normally, positive samples Smaller values ​​(stable logical structure), negative samples The value is significantly large (the logical structure is prone to collapse);

[0085] S43. Traverse the threshold and calculate the Youden index: in the interval Internally set a series of candidate thresholds For each candidate threshold Calculate the true positive rate (the proportion of correctly identified hallucinatory text) and the false positive rate (the proportion of correctly identified normal text as hallucination), and calculate the Youden index. ,in, The true positive rate, The false positive rate;

[0086] It should be noted that the Youden index is the difference between the true positive rate and the false positive rate, and is used to comprehensively evaluate diagnostic efficacy.

[0087] S44. Determine the optimal threshold: Select a threshold that maximizes the Yoden index. The largest candidate threshold is used as the final quality and safety threshold. Its expression is: This method ensures that the system intercepts as many high-risk illusions as possible while preserving normal logical leaps or rhetorical expressions to the greatest extent, avoiding false blocking of users' normal queries. In this embodiment 1, calculations using the above method show that when When set to 5.2, the Youden index Reaching peak value, therefore, setting .

[0088] At quality and safety thresholds Based on the setting of 5.2, step S3 calculates the logic collapse index. ,but The hallucination was determined to be of high risk, and a blocking operation was performed.

[0089] S5. For the original text that is determined to be a high-risk hallucination By using spectral difference information to shape the output probability values ​​(Logits) of erroneous locations in logical collapse, the path leading to collapse is suppressed during the large language model regeneration process, forcing the large language model to avoid erroneous logical paths during regeneration. The specific implementation steps are as follows:

[0090] S51. First, the system needs to identify which specific tokens caused the drastic changes in the logical graph. The method is as follows:

[0091] Difference edge location: Compare with the original directed logic graph Logic diagram of contrast with counterfactual facts Identify the set of edges with the largest weight changes or polarity reversals. For example, in this embodiment 1, the comparison reveals the edge The attribute of (hypertension → ibuprofen) changed from permitted to prohibited, on the side The weight of (ibuprofen → blood pressure) has increased;

[0092] Semantic backtracking: Utilizing alignment information from entity relation extraction, ... The relational descriptors in the text are mapped back to specific token locations in the original text. For example, in this embodiment 1, the system locks the original text. Keyword set in ={“Yes”, “Can be taken”, “Minimal impact”};

[0093] Vocabulary index mapping: Query the vocabulary of the large model to obtain... The set of Token ID indexes corresponding to each word in the text .

[0094] S52. Based on the locked error token index, construct a dictionary that is the same size as the model vocabulary. Consistent sparse mask vector ,definition Its assignment logic is as follows:

[0095] ;

[0096] In this embodiment 1, assuming that the ID of "can" in the vocabulary is 1024 and the ID of "small impact" is 3055, then the vector Bits 1024 and 3055 are set to 1, and the rest are all 0. This is equivalent to marking the logical nodes that should be muted.

[0097] S53. Introduce a penalty term to calculate the dynamic inhibition strength, the expression of which is:

[0098] ;

[0099] In the formula, The logic collapse index. This is the inhibition coefficient.

[0100] S54, at each decoding moment when the large language model initiates regeneration. The system intervenes in the output layer of the large language model, performing Logits reshaping and resampling, as follows:

[0101] S541. Obtain the original Logits: The model outputs the original prediction vector. (dimension is) );

[0102] S542. Apply penalty: Perform vector subtraction operation, the expression of which is: This means that for non-erroneous tokens (mask 0), Logits remain unchanged; for erroneous tokens (mask 1), their Logits value is significantly reduced.

[0103] S543, Probability Normalization: Normalization of the corrected vector implement The function is represented as:

[0104] ;

[0105] Will Substituting the expression into the above equation, we get:

[0106] ;

[0107] In the formula, Represented by natural constant An exponential function with base 0. This represents the index variable in the model vocabulary. Represents the transformed Logits vector The first in One component; The corrected probability distribution is the final output after Softmax normalization. In this new probability distribution, the probability of words that cause logical collapse (such as "can") is suppressed to an extremely low value (close to 0), while the probability of logically sound words (such as "use with caution") is relatively increased, thus guiding the sampling strategy to avoid the illusion path.

[0108] S544, Sampling Output: Based on the probability distribution calculated above. Sampling is performed, and since the probability of erroneous paths (such as "can be taken") has been suppressed to near 0, the large language model will be forced to explore other high-probability paths (such as "suggest consultation", "use with caution", etc.), thereby achieving logical self-correction and generating new text without the need for an external knowledge base;

[0109] S55. Use the new text generated after Logits shaping as the new input sequence. The optimization count N is incremented by 1 each time new text is generated. If the optimization count is greater than or equal to the preset maximum count, the process ends and an error is output; otherwise, steps S1 to S4 are called to perform a second logic collapse index calculation and verification. The logic collapse index of the newly generated text is then used as the input sequence. Less than the quality and safety threshold At that time, the output is displayed to the user.

[0110] It should be noted that the inhibition coefficient The system employs a self-normalization method based on the current output state of the large language model for dynamic calculation. Specifically, the system obtains the current decoding time. The original logits vector The system identifies the element with the largest value (i.e., the preferred token currently generated by the large language model) and the element with the second largest value (i.e., the second-best token), calculates the difference between them, and constructs an adaptive calculation formula based on the principle of minimum effective intervention: ;

[0111] In the formula, Indicates time The inhibition coefficient, This indicates the preferred token generated by the large language model. This represents the token generated by the current second selection in the large language model. The logic collapse index. It is a small constant, usually taken as 10. -6 This is used to break the critical balance and ensure that the Logits value of the original preferred token after the penalty is applied is slightly lower than that of the original secondary token.

[0112] In this embodiment 1, by combining the logic collapse index and the real-time Logits difference to dynamically calculate the suppression coefficient, a penalty sufficient to change the output order is imposed on the illusion path, forcing the large language model to achieve logical correction without destroying the fluency of the sentences, thereby eliminating the high-risk illusions in the original text. Example

[0113] As a second embodiment of the present invention, such as Figure 2 As shown in Example 1, this example also discloses a text quality processing system based on a large language model, which specifically includes: a logic graph spectrum analysis module, a counterfactual interference generation module, a logic collapse assessment module, a quality judgment module, and a self-correcting optimization module.

[0114] The logic graph spectrum analysis module is used to receive prompt words input by the user and call a large language model to generate original generated text; extract key entities and relationships from the original generated text using an entity relation extraction model to construct a directed logic graph; calculate the degree matrix and adjacency matrix of the directed logic graph to construct a Laplacian matrix, and perform eigenvalue decomposition on the Laplacian matrix to obtain the eigenvalue sequence of the original graph.

[0115] The counterfactual interference generation module is used to calculate the load-bearing coefficient of each entity in the original text based on the attention gradient, and select the entity with the highest load-bearing coefficient as the logical load-bearing wall; locate the core attribute edge of the logical load-bearing wall and generate a logically mutually exclusive description, construct interference prompt words containing counterfactual assumptions and inject them into the large language model, and forcibly guide the generation of counterfactual comparison text.

[0116] The logic collapse assessment module is used to construct a contrast logic graph of the counterfactual text and calculate the eigenvalue sequence of its Laplace matrix; based on the difference between the eigenvalue sequence of the original graph and the eigenvalue sequence of the contrast logic graph, and combined with the logic polarity reversal, the logic collapse index, which characterizes the stability of the text's logical structure, is calculated.

[0117] The quality judgment module is used to preset a quality and safety threshold, compare the logic collapse index with the quality and safety threshold; if the logic collapse index is greater than or equal to the quality and safety threshold, the generated text is determined to be a high-risk hallucination and a blocking operation is performed; if the logic collapse index is less than the quality and safety threshold, it is determined to be a low-risk hallucination and the text is output.

[0118] The self-correcting optimization module is used to identify the set of edges with the largest weight changes and polarity reversals when comparing the original directed logic graph with the counterfactual contrast logic graph for texts judged as high-risk hallucinations. It also maps relational descriptors back to the erroneous token index positions in the original text. A sparse mask vector is constructed, and at the decoding time of the large language model regeneration, the suppression strength is dynamically calculated using the logic collapse index. The Logits values ​​corresponding to the erroneous tokens are penalized, shaped, and normalized, forcing the large language model to avoid erroneous logic paths and resample to generate new text.

[0119] In the specific implementation of Implementation 2 above, the logic graph spectrum analysis module serves as the system's perception entry point. It receives user-input prompts and calls the large language model to generate the original text. Subsequently, it uses a built-in entity relationship extraction model to scan the original text, extracting key entities and relationships to construct a directed logic graph. Furthermore, it constructs a Laplacian matrix for eigenvalue decomposition, thereby obtaining the eigenvalue sequence of the original graph. This module transforms unstructured natural language text into computer-processable digital graph features. Through spectrum analysis, it maps the inherent logical structure of the text into a quantified spectral sequence, establishing a mathematical benchmark for measuring logical stability. This allows the system to extract features from text logic without relying on an external knowledge base. The counterfactual interference generation module receives the original text information, calculates the load-bearing coefficient of each entity in the text based on attention gradients, identifies the logical load-bearing walls that play a key supporting role in the validity of the logic, and then performs semantic flipping on the core attributes of the entity. It constructs interference prompts containing strong assumptions and injects them back into the large language model, guiding the model to generate contrast text under counterfactual conditions. This module generates targeted counterfactual text by locating load-bearing walls and applying counterfactual attacks. This effectively exposes potential logical flaws, providing comparative material for subsequent difference assessments. The logic collapse assessment module constructs and analyzes the generated counterfactual text to obtain a sequence of characteristic values ​​from the corresponding logic graph. It then compares the differences in characteristic value sequences between the original text and the counterfactual text, and considers whether logical polarity has reversed, to calculate a logic collapse index. This module achieves a leap from qualitative analysis to quantitative assessment. The logic collapse index intuitively quantifies the degree of logical distortion of the original text when subjected to counterfactual attacks, sensitively capturing illusory content that is linguistically grammatically correct but logically weak, providing objective data support for risk assessment. The quality judgment and control module presets a quality and safety threshold and compares the real-time calculated logic collapse index with this threshold. When the logic collapse index exceeds the threshold, the generated text is judged as a high-risk illusion and blocked; otherwise, it is judged as low-risk and output normally. This module establishes an automated quality and safety defense line, automatically segmenting text based on its logical robustness. It effectively blocks generated content with serious logical fallacies while allowing normal rhetorical expressions, balancing system security and usability. The self-correcting optimization module, targeting high-risk blocked text, compares the original logic graph with a control logic graph to pinpoint the key error paths leading to logical collapse. During the decoding process of model regeneration, it dynamically adjusts the suppression strength using a logic collapse index, shaping and suppressing the output probability values ​​(Logits) at error locations, forcing the model to avoid known erroneous logical branches during resampling.This module, through Logits shaping, can guide the model to explore new, logically consistent generation paths using its own logical verification information without relying on external references, thereby achieving automatic error correction and quality improvement of the generated content. Example

[0120] As a third embodiment of the present invention, based on embodiment 1, this embodiment also discloses an entity relation extraction model for realizing the extraction of entity relations from raw text. Extract key entity set With relation set Construct a directed logic graph The entity relation extraction model consists of three parts: a context encoding layer, a subject tagging layer, and an object tagging layer under specific relations. Specifically:

[0121] Context encoding layer: Input is the original generated text Convert it into a token sequence Using a pre-trained BERT-Base-Chinese model as the encoder, deep semantic features are extracted:

[0122] ;

[0123] in, Indicates the first The context feature vector of each token, The dimension of the hidden layer (e.g., 768).

[0124] Subject Tagging Layer (First-Layer Pointer Network): This layer identifies all possible logical subjects in the text. Essentially, it involves two binary classification tasks: predicting whether each token represents the "start" or "end" position of the logical subject. For the token, the probability of it being predicted as the start or end of the subject is... They are respectively:

[0125] ;

[0126] ;

[0127] in, It is the Sigmoid activation function. and For trainable weights and biases. Set a threshold. ,like If so, then it is marked as 1.

[0128] Object tagging layer under specific relations (second-layer pointer network): for each candidate subject extracted from the first layer (Assuming its span is) arrive This layer is responsible for specific logical relationships. Under constraints such as "cause and effect," "condition," and "contrast," the corresponding object is found. To incorporate subject information, max pooling is first used to calculate the subject's feature vector. :

[0129] ;

[0130] Then, subject characteristics Features of each token in the text sequence To merge, input into the relationship In the decoder:

[0131] ;

[0132] ;

[0133] This structure means that, for The model will output in parallel based on predefined logical relationships. A sequence of probability distributions is used to accurately capture one-to-many or overlapping logical relationships such as "A causes B, and A is better than C".

[0134] Loss function definition: The training objective of the model is to minimize the joint loss of subject and object tags. The binary cross-entropy loss function (BCE) is used. Overall loss function. Defined as:

[0135] ;

[0136] Among them, subject loss for:

[0137] ;

[0138] Object loss It is necessary to iterate through all extracted valid subjects. and all predefined relationships :

[0139] ;

[0140] In the formula, For true labels (0 or 1).

[0141] Logical targeted training data construction:

[0142] Data sources: selected from publicly available legal document judgment datasets (CAIL), medical guideline documents, and logic reasoning competition datasets.

[0143] Relationship Definition: Unlike conventional entity relationships such as "place of birth" and "founder," this embodiment predefines a set of core logical relationships designed to serve spectral analysis, including but not limited to:

[0144] Strong causality: The weight is set to 1.0, and it is used to build the main framework.

[0145] Conditional constraint: The weight is set to 0.8, corresponding to the "only if...then..." sentence structure.

[0146] Turning Point / Opposition: The weight is set to -1.0 or a special phase, corresponding to the negative connection in the Laplace matrix, and is used to capture logical polarity reversals.

[0147] Attribute Attribution: The weight is set to 0.5, which is used for fine-grained entity description.

[0148] Data augmentation: Declarative texts are rewritten into complex long and difficult sentences using high-quality large language models (such as GPT-4), and the above logical triples are automatically labeled. After manual sampling and proofreading, the data is added to the training set to improve the model's ability to parse complex syntactic structures (such as inversion and multiple negations).

[0149] Training parameter settings:

[0150] Optimizer: AdamW optimizer is used. Learning rate: The learning rate of the BERT layer is set to... The learning rate for the remaining layers is set to Warm-up strategy: Linear warm-up for the first 10% of steps. BatchSize: 32. Dropout: 0.1 to prevent overfitting.

[0151] Through the above architecture, the model can accurately output a list of (Subject, Relation, Object) triples from unstructured text. These triples are directly mapped to the directed logic graph described in step S12 of Example 1. The nodes and edges, where the type of Relation directly determines the adjacency matrix. The weight values ​​in the matrix provide a high-quality topological foundation for subsequent Laplace matrix construction and eigenvalue decomposition.

[0152] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0153] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A method for processing quality of generated text based on a large language model, characterized in that, include: The original text is generated based on the prompts entered by the user, and a logical topology graph and a Laplace matrix are constructed. The Laplace matrix is ​​then decomposed into eigenvalues ​​to obtain the eigenvalue sequence of the original graph. Based on the attention gradient calculation, the load-bearing coefficient of each entity in the original text is analyzed and identified. The logical load-bearing walls are then analyzed and counterfactual semantic flipping is implemented and the large language model is guided to generate counterfactual contrast text. Construct a comparison logic diagram and Laplace matrix eigenvalue sequence of the counterfactual text, and calculate the logic collapse index by combining the eigenvalue differences between the original text and the counterfactual text. The quality of the generated text is determined by comparing the logic collapse index with a preset quality and safety threshold. For text that fails the quality assessment, a blocking operation is performed. At the same time, the output probability values ​​of the logically collapsed error locations are shaped and sampled to generate new text. The weight coefficient of each entity in the original text is calculated based on the attention gradient The expression is: ； In the formula, For the number of attention heads, This is the aggregate semantic representation vector of the entire sentence text. For entities exist Feature vector representation in each attention head; Select the load-bearing coefficient The tallest entity acts as a logical load-bearing wall; The expression for calculating the logic collapse index is as follows: ； In the formula, The logic collapse index. To generate text based on the original Directed logic graph, For counterfactual comparison text The comparison logic diagram, For the first One eigenvalue; This is the polarity reversal penalty coefficient; This is a logical reversal flag; it takes a value of 1 if the main conclusion changes from affirmative to negative, and 0 otherwise. The generation of new text specifically includes: Compare with the original directed logic graph Logic diagram of contrast with counterfactual Identify the set of edges with the largest weight changes and polarity reversals, and map relation descriptors back to the set of erroneous word indexes in the original text; Construct a sparse mask vector, assigning a value of 1 to the position corresponding to the erroneous word and 0 to the rest; At the decoding stage of the large language model regeneration, the original prediction vector output by the large language model is obtained, and the original prediction vector is corrected using the logistic collapse index to obtain the corrected vector. The modified vector is subjected to Softmax normalization to obtain a probability distribution, and new text is generated by sampling based on the probability distribution.

2. The text quality processing method based on a large language model according to claim 1, characterized in that, The construction of the logical topology graph and Laplace matrix specifically includes: generating text from the original text using an entity relationship extraction model. Extract key entity set With relation set Construct a directed logic graph ; Calculate the directed logic graph degree matrix Adjacency Matrix Construct the Laplace matrix .

3. The text quality processing method based on a large language model according to claim 2, characterized in that, The eigenvalue sequence of the original graph is obtained by solving for the roots of the characteristic polynomial, specifically including: [the following steps are taken to obtain the Laplace matrix]. Perform eigenvalue decomposition and solve for the characteristic polynomial. The root is used to obtain the original spectral feature value sequence arranged in ascending order. ;in, This represents determinant calculation. It is the identity matrix. Then it represents the eigenvalue.

4. The method for improving the quality of generated text based on a large language model according to claim 1, characterized in that, The analysis and identification of logical load-bearing walls and the implementation of counterfactual semantic flipping specifically include: Locate the core attribute edge of the logical load-bearing wall in the logical topology graph, and use a semantic adversarial generator to generate a logically mutually exclusive description of the core attribute edge; Construct distractor words containing counterfactual assumptions It is then injected into a large language model, forcibly guiding the model to generate counterfactual contrast text. .

5. The method for improving the quality of generated text based on a large language model according to claim 1, characterized in that, The determination of the quality of the generated text specifically includes: setting a quality safety threshold. ; If the logic collapses index The hallucination was determined to be of high risk, and a blocking operation was performed. If the logic collapses index If the illusion is determined to be low-risk, the text will be output to the user.

6. The method for improving the quality of generated text based on a large language model according to claim 5, characterized in that, The quality and safety threshold The Yoden index was used for optimization, specifically including: constructing a logistic calibration dataset containing positive and negative samples, calculating the logistic collapse index for each sample, and iterating through candidate thresholds. Calculate the Youden index ,in, The true positive rate, The false positive rate; select the value that makes the Youden index... The largest candidate threshold is used as the quality and safety threshold. .

7. A system for improving the quality of generated text based on a large language model, used to execute the method for improving the quality of generated text based on a large language model as described in any one of claims 1 to 6, characterized in that, include: The logic graph spectrum analysis module is used to generate the original text based on the user prompts, extract entity relationships to construct a directed logic graph and Laplace matrix, and perform eigenvalue decomposition to obtain the original graph eigenvalue sequence. The counterfactual interference generation module is used to identify logical load-bearing walls in the original text based on attention gradient, perform counterfactual flipping on their core attributes to construct interference prompt words, and guide the large language model to generate counterfactual comparison text. The logic collapse assessment module is used to construct a comparison logic diagram and feature value sequence of the counterfactual comparison text, and to quantitatively calculate the logic collapse index by combining the differences in the feature value sequence of the original graph. The quality assessment module is used to compare the logic collapse index with a preset quality safety threshold and determine the quality of the generated text based on the comparison result. The self-correcting optimization module is used to suppress and resample the output probability values ​​of logically collapsed erroneous paths during the model regeneration process, and generate new text.

Images