A large-scale model optimization method, apparatus, computer equipment, and storage medium based on tiered cascading circuit breaker rewards.

Abstract

This application proposes a large-scale model optimization method, apparatus, computer device, and storage medium based on a hierarchical cascading circuit breaker reward system. The method includes: performing format conformity checks on candidate texts; if the check fails, a circuit breaker is triggered, a first negative reward value is assigned to the candidate text, and the text state of the candidate text is marked as a first circuit breaker state; if the check succeeds, a factual consistency check is performed on the candidate text; if the check fails, a circuit breaker is triggered, a second negative reward value is assigned to the candidate text, and the text state of the candidate text is marked as a second circuit breaker state; if the check succeeds, a multi-dimensional score is applied to the candidate text to obtain a final positive reward value, and the text state of the candidate text is marked as a passed state; based on the text states and reward values ​​of multiple candidate texts, model optimization is performed on the model to be optimized to obtain the final model. This application can improve the reliability of the model in generating texts with high requirements for format conformity and factual accuracy.

Description

Technical Field

[0001] This application relates to the field of model optimization technology, specifically to a large model optimization method, apparatus, computer equipment, and storage medium based on hierarchical cascaded circuit breaker rewards. Background Technology

[0002] With the development of large language model technology, automatically generating official documents using large models has become a research hotspot. However, official documents have extremely high requirements for format standardization and factual accuracy, and existing technical solutions have significant shortcomings.

[0003] Existing reward mechanisms typically perform a comprehensive parallel evaluation of candidate output texts, requiring quality assessment across all dimensions, leading to a significant waste of computational resources. Furthermore, existing methods weight multi-dimensional scores into a single total score, resulting in a flat and ambiguous signal. The model cannot distinguish the priority differences between formatting errors and factual errors, leading to slow convergence. More critically, existing mechanisms lack stringent constraints, potentially allowing models to sacrifice factual accuracy for literary merit, making it difficult to eliminate the risk of "illusions."

[0004] Therefore, existing technologies lack pre-screening and circuit breaker mechanisms, fail to reflect differences in error severity in reward signals, and lack mechanisms to enforce key hard constraints, thus failing to meet the high reliability requirements of official documents. Summary of the Invention

[0005] In view of this, this application proposes a large model optimization method, apparatus, computer equipment and storage medium based on hierarchical cascading circuit breaker rewards, in order to solve the problem of low reliability of large models in related technologies when generating text with high requirements for format standardization and factual accuracy.

[0006] The first aspect of this application proposes a large-scale model optimization method based on hierarchical cascaded circuit breaker rewards, the method comprising: Obtain multiple candidate texts; the multiple candidate texts are output by the strategy model to be optimized based on the text task instructions and the original text; For any candidate text among the plurality of candidate texts, perform a format standardization check on the candidate text. If the check fails, trigger a circuit breaker, assign a first negative reward value to the candidate text, and mark the text status of the candidate text as the first circuit breaker state. If the verification is successful, the candidate text is checked for factual consistency. If the verification fails, a circuit breaker is triggered, a second negative reward value is assigned to the candidate text, and the text status of the candidate text is marked as the second circuit breaker state. The absolute value of the second negative reward value is greater than the first negative reward value. If the verification is successful, the candidate text is scored in multiple dimensions to obtain the final positive reward value of the candidate text, and the text status of the candidate text is marked as passed. Based on the text states and reward values ​​of the multiple candidate texts, the model to be optimized is optimized to obtain the final model.

[0007] This application's embodiment employs a tiered, cascading circuit breaker architecture (i.e., format conformity verification is performed first, followed by factual consistency verification, and finally multi-dimensional scoring), transforming the traditional parallel evaluation into a serial, checkpoint-based verification process. Candidate texts are eliminated during the format conformity verification stage, avoiding further factual consistency verification and multi-dimensional scoring, thus preventing ineffective in-depth evaluation of clearly substandard outputs. This early elimination mechanism saves significant computing resources, allowing the system's computing power to concentrate on the refined evaluation of a few high-quality candidates, resulting in a significant improvement in overall review efficiency.

[0008] This application's embodiments present extreme differentiation through reward value settings: formatting errors are assigned a first negative reward value, factual errors are assigned a second negative reward value with a larger absolute value, and candidate texts that pass receive positive rewards. This significant disparity creates a strong contrast in the relative advantage calculation results, allowing the model to clearly perceive the hierarchy of error severity. The strongest negative signal of factual errors creates a high-pressure line effect, prompting the model to prioritize learning to avoid formatting and factual errors before gradually optimizing soft quality dimensions. The black-and-white signal design eliminates the ambiguity of traditional mixed rewards, resulting in a clear training direction and faster convergence speed.

[0009] In this embodiment of the application, the format conformity check of the candidate text includes: The candidate text is parsed in a structured manner to obtain multiple structured fields, and the multiple structured fields are verified to conform to the preset hierarchical style specifications; the multiple structured fields include article title, entry title, body text, and data source; The candidate text is template matched using multiple regular expressions to determine its key format and content features. These regular expressions include a first regular expression for validating the format of the entry title, a second regular expression for validating the standardization of number usage, and a third regular expression for validating the accuracy of the full name of the organization. If the multiple structured fields do not conform to the preset hierarchical style specification, or if the template matching of any regular expression fails, then the candidate text validation is determined to have failed.

[0010] In this embodiment of the application, the factual consistency check of the candidate text includes: Extract a first entity and a first value from the candidate text, and generate a set based on the first entity and the first value; Filter the second entity and the second value that match the text task instruction from the preset knowledge index, and form a source set based on the second entity and the second value; Calculate the difference between the generated set and the source set. If the difference is not empty, the factual consistency check of the candidate text is determined to have failed.

[0011] In this embodiment of the application, the method further includes: Based on the domain knowledge corresponding to the text task instruction, a logical check is performed on the first value in the generated set. If the logical check fails, it is determined that the factual consistency check of the candidate text has failed.

[0012] This application's embodiments employ a circuit breaker mechanism based on factual consistency checks, granting the highest veto power over factual accuracy. Any candidate text that differs from the source set, or fails logical verification, triggers the circuit breaker and incurs the most severe penalty. This zero-tolerance mechanism prevents the model from sacrificing factual accuracy for the sake of elegant writing, fundamentally suppressing the risk of content illusion. A state marking mechanism ensures that only formatted and factually accurate candidates enter the quality optimization stage, enabling the final model output to adapt to application scenarios with high requirements for content accuracy and formatting compliance.

[0013] In this embodiment of the application, the candidate text is scored in multiple dimensions to obtain the final positive reward value of the candidate text, including: A basic positive reward value is assigned to candidate texts that pass both the format conformity check and the factual consistency check. The candidate texts are scored on multiple dimensions, including logicality, objectivity, and conciseness. The scores on the multiple dimensions are weighted and summed according to the preset weights corresponding to each dimension to obtain the weighted sum result; The weighted summation result is superimposed with the basic positive reward value to generate the final positive reward value of the candidate text.

[0014] In this embodiment of the application, the final model is obtained by optimizing the strategy model to be optimized based on the text state and reward value of the plurality of candidate texts, including: Calculate the average reward for the multiple candidate texts; For any candidate text among the plurality of candidate texts, a relative advantage value is calculated based on the reward value of the candidate text and the average reward value; a positive relative advantage value indicates that the text generation level of the candidate text is higher than the average level; a negative relative advantage value indicates that the text generation level of the candidate text is lower than the average level. A policy gradient objective function is constructed based on the relative advantage values ​​of the multiple candidate texts; the policy gradient objective function is used to increase the generation probability of candidate texts with positive relative advantage values, while decreasing the generation probability of candidate texts with negative relative advantage values. The final model is obtained by performing multiple rounds of optimization on the policy model to be optimized based on the policy gradient objective function.

[0015] In this embodiment of the application, the text task instruction is used to instruct the strategy model to be optimized to generate the target document based on the original text.

[0016] An embodiment of the second aspect of this application provides a large model optimization apparatus based on hierarchical cascaded circuit breaker rewards, comprising: The candidate text acquisition module is used to acquire multiple candidate texts; the multiple candidate texts are output by the strategy model to be optimized based on the text task instructions and the original text. The format standardization verification module is used to perform format standardization verification on any candidate text among the plurality of candidate texts. If the verification fails, a circuit breaker is triggered, a first negative reward value is assigned to the candidate text, and the text status of the candidate text is marked as the first circuit breaker state. The fact consistency verification module is used to perform fact consistency verification on the candidate text if the verification is successful, and to trigger a circuit breaker if the verification fails, assign a second negative reward value to the candidate text, and mark the text status of the candidate text as a second circuit breaker state; the absolute value of the second negative reward value is greater than the first negative reward value; The multi-dimensional scoring module is used to score the candidate text in multiple dimensions if the verification is successful, obtain the final positive reward value of the candidate text, and mark the text status of the candidate text as passed. The model optimization module is used to optimize the strategy model to be optimized based on the text state and reward value of the multiple candidate texts to obtain the final model.

[0017] An embodiment of the third aspect of this application provides a computer device including a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the large model optimization method based on hierarchical cascaded circuit breaker rewards described in the first aspect.

[0018] An embodiment of the fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the large model optimization method based on hierarchical cascaded circuit breaker rewards described in the first aspect above.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a large model optimization method based on hierarchical cascaded circuit breaker rewards provided in an embodiment of this application is shown. Figure 2 A flowchart illustrating another large model optimization method based on hierarchical cascaded circuit breaker rewards provided in an embodiment of this application is shown. Figure 3 A flowchart illustrating the hierarchical cascaded circuit breaker reward architecture provided in an embodiment of this application is shown. Figure 4 This illustration shows a comparison of execution paths for different candidate texts provided in an embodiment of this application; Figure 5 This illustration shows a schematic diagram of a large model optimization device based on hierarchical cascaded circuit breaker rewards according to an embodiment of this application; Figure 6 This illustration shows a schematic diagram of the structure of a computer device according to an embodiment of this application; Figure 7 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0021] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0022] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0023] The technical scenarios involved in the embodiments of this application are described below.

[0024] With the rapid development of artificial intelligence technology centered on Large Language Models (LLMs), relevant editorial units are actively exploring the application of large models to improve work efficiency and quality in official document writing scenarios, where the accuracy of content and the standardization of format are extremely demanding. Compared with conventional technical reports or legal documents, the writing of official documents, which has extremely high requirements for the accuracy of content and the standardization of format, has its own unique complexity and challenges. Its format, style, and content elements are subject to strict and detailed regulations, with numerous rules that allow no deviation, and extremely high requirements for the authority of the content and the seriousness of historical records. However, when using mainstream reinforcement learning algorithms such as GRPO (Group Relative Policy Optimization) to optimize the model, many pain points still exist: 1. Inefficient reward calculation. Existing reward mechanisms typically perform a comprehensive quality assessment on every candidate output generated by the model. Even if some outputs are already substandard at the most basic level of the preset document writing style (such as article structure and item titles), the system will still consume a lot of computing power to evaluate higher-order quality dimensions such as fluency and logic, resulting in a large waste of valuable computing resources.

[0025] 2. Ambiguous training signals and slow convergence. Reward models or functions tend to weight scores from multiple dimensions such as format, factual accuracy, and relevance into a single total score. This ambiguous signal makes it difficult for the model to clearly distinguish the priority of errors. For example, format errors versus factual errors; or a logically flawed statement that completely deviates from the preset document writing style may receive similar scores as an output that is factually accurate but has an incorrectly formatted chapter title. This hinders the model from quickly learning and adhering to those "must-not-be-wrong" hard rules, resulting in slow training convergence.

[0026] 3. The "illusion" of the model is difficult to eradicate. The generation of content that does not conform to the facts is a core challenge for current large-scale models. In official document writing scenarios where the accuracy of content and the standardization of format are extremely demanding, any factual error is unacceptable. While existing reward mechanisms can punish factual errors, they have not formed a "high-voltage line" mechanism with the highest veto power. The model still has the probability of sacrificing factual accuracy in pursuit of bonuses in other dimensions such as elegant writing style.

[0027] This application provides a large model optimization method based on hierarchical cascaded circuit breaker rewards, aiming to solve the above-mentioned technical problems: 1. By constructing a hierarchical and cascading "circuit breaker" reward architecture (i.e., first performing format standardization checks on candidate texts, and only performing factual consistency checks after passing the checks), the reward calculation is transformed from a parallel "package" evaluation into a serial verification process with checkpoints. This solves the problem of wasted computing resources caused by ineffective in-depth evaluation of unqualified outputs and provides the model with a clear, priority-based learning path.

[0028] 2. By introducing a "fact consistency gating" (i.e., performing fact consistency checks on the candidate text) based on set difference (i.e., the difference between the generated set and the source set) and logical verification, the model-generated content is subjected to "zero tolerance" illusion detection, thereby solving the technical problem that the existing reward mechanism cannot prioritize "fact accuracy" and thus leads to insufficient content reliability.

[0029] 3. By deeply integrating the clear and strong negative reward signal generated by the circuit breaker mechanism with the GRPO algorithm, a clear and black-and-white training guide is provided for the policy model (i.e., the final model is obtained by optimizing the policy model based on the text state and reward value of the multiple candidate texts), thereby solving the problem of fuzzy traditional mixed reward signals, which leads to slow model convergence and unclear training direction.

[0030] According to an embodiment of this application, a large model optimization method based on hierarchical cascaded circuit breaker rewards is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] This embodiment provides a large model optimization method based on hierarchical cascaded circuit breaker rewards, which can be used as a review agent in a large model optimization system based on hierarchical cascaded circuit breaker rewards. Figure 1 This is a flowchart of a large model optimization method based on tiered cascaded circuit breaker rewards according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain multiple candidate texts.

[0032] Specifically, such as Figure 2 As shown, multiple candidate outputs (i.e., the aforementioned candidate texts) can be obtained from the generative agent in a large model optimization system based on hierarchical cascading circuit breaker rewards. The generative agent, acting as the "generative agent" of the policy model to be optimized, generates a set (e.g., N) of candidate output texts based on the given input (such as raw data or text task instructions).

[0033] More specifically, a pre-set reinforcement learning algorithm (such as Group Relative Policy Optimization (GRPO) algorithm) can be used to optimize the model of the policy to be optimized.

[0034] In some specific embodiments, text task instructions are structured input information that directs the strategy model to perform a specific writing task. In official document writing scenarios where the accuracy of content and the standardization of format are extremely important, the text task instructions clearly inform the model of the specific document to be written, the topic of the item, and the formatting requirements that must be followed.

[0035] In this embodiment, the text task instruction includes three core elements: First, a task type identifier, explicitly specifying "a specific type in an official document writing scenario with extremely high requirements for content accuracy and format standardization," causing the model to switch to the corresponding generation mode; second, target document and item information, including broad categories such as "Economic Review" and "Social Undertakings," and specific item titles such as "Industrial Development" and "Education Undertakings"; and third, a format constraint declaration, mandating that the output must conform to the preset official document writing style, including hard requirements such as using square brackets in the title, indicating data sources, and using objective and concise language.

[0036] Step S102: For any candidate text among the plurality of candidate texts, perform a format standardization check on the candidate text. If the check fails, trigger a circuit breaker, assign a first negative reward value to the candidate text, and mark the text status of the candidate text as the first circuit breaker state.

[0037] Specifically, the first circuit breaker state indicates that the candidate text failed the format specification check. The format specification check is used to quickly filter out all candidate texts that do not conform to the basic format and specifications.

[0038] More specifically, triggering the circuit breaker mechanism if the verification fails can be understood as follows: the candidate text fails the format specification verification, all subsequent review processes are suspended, and it will no longer enter the factual consistency verification and multi-dimensional scoring.

[0039] In some specific embodiments, the candidate text is subjected to format conformity verification, including steps S1021-S1023: Step S1021: Perform structured parsing on the candidate text to obtain multiple structured fields, and verify whether the multiple structured fields conform to the preset hierarchical style specification.

[0040] Specifically, the multiple structured fields include article title, entry title, body text, and data source.

[0041] In some specific implementations, for official document writing scenarios with extremely high requirements for content accuracy and format standardization, the following fields are extracted through structured parsing: First, the title field, corresponding to the major category of the official document, such as economic overview, social undertakings, cultural construction, etc., this field must be enclosed in square brackets. Second, the item title field, corresponding to the specific item name, such as industrial development, education, medical and health care, etc. Third, the body text field, containing objectively stated factual content, requiring accurate data and concise expression. Fourth, the data source field, indicating the authoritative source of all values, which must begin with a specific prefix and be enclosed in square brackets.

[0042] In some specific embodiments, a predefined hierarchical style specification defines the existence and format requirements for the aforementioned structured fields. The existence requirements stipulate that the title, body text, and data source are mandatory fields, while the entry title is optional. The format requirements stipulate that the title must be enclosed in square brackets, containing two to eight Chinese characters, and must not contain numbers or punctuation. The body text must be in objective declarative sentences, and emotional punctuation such as exclamation marks and question marks is prohibited. The data source must use a fixed format, fully indicating the institution name and document title.

[0043] In some specific embodiments, such as Figure 2 As shown, the review agent can obtain structured parsing rules from the rule / knowledge base management module, perform structured parsing on candidate text using these rules to obtain multiple structured fields, and verify whether these structured fields conform to the preset hierarchical style specifications.

[0044] In this embodiment of the application, if a required field is found to be missing or the hierarchical structure is disordered during the verification process, it is determined that the structured parsing has failed, and the process immediately jumps to the circuit breaker process. The candidate text is assigned a first negative reward value (e.g., -10), and the text status of the candidate text is marked as the first circuit breaker state (e.g., CIRCUIT_BREAK_LEVEL_1).

[0045] Step S1022: Template matching is performed on the key format and content features of the candidate text using multiple regular expressions.

[0046] Specifically, the multiple regular expressions include a first regular expression for validating the format of the entry title, a second regular expression for validating the standardization of number usage, and a third regular expression for validating the accuracy of the full name of the organization. The review agent can obtain these multiple regular expressions from the rule / knowledge base management module.

[0047] In some specific embodiments, verifying the format of an entry title can be understood as verifying whether the format of the entry title conforms to the standard requirement of square brackets plus Chinese characters for the title; verifying the standardization of the use of numbers can be understood as checking whether the use of numbers uses Arabic numerals and is accompanied by standard units; verifying the accuracy of the full name of an organization can be understood as checking whether the name of the organization uses the full name rather than the abbreviation.

[0048] Step S1023: If the multiple structured fields do not conform to the preset hierarchical style specification, or if the template matching of any regular expression fails, then the candidate text verification is determined to have failed.

[0049] Specifically, if multiple structured fields obtained by structured parsing do not conform to the preset hierarchical style specification, or if the template matching of any regular expression fails, the candidate text verification is determined to have failed, thereby triggering the circuit breaker mechanism, assigning a first negative reward value to the candidate text, and marking the text status of the candidate text as the first circuit breaker state.

[0050] Step S103: If the verification is successful, the candidate text is checked for factual consistency. If the verification fails, the circuit breaker is triggered, a second negative reward value is assigned to the candidate text, and the text status of the candidate text is marked as the second circuit breaker state.

[0051] Specifically, the absolute value of the second negative reward value (e.g., -20) is greater than that of the first negative reward value (e.g., -10).

[0052] Specifically, the second circuit breaker state indicates that the candidate text passed the format conformity check but failed the factual consistency check. The factual consistency check is to eliminate "illusions" and ensure that the content of the candidate text is true and accurate.

[0053] More specifically, triggering the circuit breaker mechanism if the verification fails can be understood as follows: the candidate text passes the format specification verification but fails the factual consistency verification, thus suspending all subsequent review processes and no longer entering the multi-dimensional scoring.

[0054] In some specific embodiments, fact consistency verification is performed on the candidate text, including steps S1031-S1033: Step S1031: Extract the first entity and the first value from the candidate text, and form a generation set based on the first entity and the first value.

[0055] Specifically, the first entity includes, but is not limited to, time entities (e.g., 2023), location entities (e.g., XX city), and institutional entities (e.g., the municipal statistics bureau); the first value includes, but is not limited to, absolute values ​​(e.g., 520 billion yuan), relative ratios (e.g., 5.2%), and structural proportions (e.g., 45.0%).

[0056] In some specific embodiments, a first entity can be paired with a first numerical value to form factual units with semantic association. For example, the first factual unit is the year 2023 associated with the location XX city and the GDP value of 520 billion yuan. The second factual unit is the year 2023 associated with the growth rate value of 5.2%. The third factual unit is the primary industry value of 26 billion yuan associated with the proportion of 5.0%. The set of these factual units constitutes the generated set.

[0057] In some specific embodiments, the extraction method of the first entity and the first value is not specifically limited in this application, such as: rule template matching extraction method, pre-trained semantic model extraction method; In some specific embodiments, the first entity and the first value are extracted through rule template matching, including: pre-constructing an entity regular expression rule library and a value regular expression rule library; wherein, the entity regular expression rule library is configured with feature matching templates for time, location, and organization-type entities, and the value regular expression rule library is configured with format matching templates for absolute values, relative ratios, and structural proportions. Candidate text is traversed and matched sentence by sentence using regular expressions to filter out the first entity and the first value that meet the features. Simultaneously, semantic pairing of entities and values ​​is completed based on syntactic position and proximity association principles to construct fact units.

[0058] Specifically, let's take the candidate text statement to be processed as an example: "In 2023, the XX Municipal Bureau of Statistics announced that the total regional economic output was 520 billion yuan, with a year-on-year growth rate of 5.2%, and the service industry accounted for 45.0% of the economy".

[0059] Time entity template: Matches the format [four-digit year + year], hit: 2023; Location entity template: Matches the [province, city, county / district] geographical naming feature, hit: XX city; Organizational entity template: Matches organizational suffixes such as [Bureau, Commission, Office, Center, Institute, Research Institute], and hits: XX Municipal Bureau of Statistics.

[0060] In some specific embodiments, the first entity and the first numerical value are extracted through a pre-trained semantic model, including: using a named entity recognition pre-trained model for structured text extraction, inputting candidate text into the model for sequence labeling; identifying first entities of the time, location, and organization categories through the model's built-in entity category labels, and identifying first numerical values ​​of the absolute value, relative ratio, and structural proportion categories through the model's built-in numerical value category labels. Then, based on the model's semantic dependency analysis results, an association mapping between entities and numerical values ​​is established, and each fact unit is automatically aggregated and generated.

[0061] Specifically, let's take the candidate text statement to be processed as an example: "In 2023, the XX City Statistics Bureau released data showing that the city's GDP reached 520 billion yuan, a year-on-year increase of 5.2%, with the primary industry accounting for 45.0%".

[0062] The model performs sequence labeling on the entire sentence, assigning category labels to each character and word; Automatic identification based on entity category tags: Time entity: 2023; Location entity: XX City; Institution entity: Municipal Bureau of Statistics.

[0063] Step S1032: Select a second entity and a second value that match the text task instruction from the preset knowledge index, and form a source set based on the second entity and the second value.

[0064] Specifically, authoritative facts relevant to the current writing task are retrieved from a pre-built knowledge index to form a set of validation benchmarks.

[0065] More specifically, the pre-built knowledge index is constructed in advance by the source material preprocessing and indexing modules, containing various data structures, such as: a knowledge graph storing entity relationship triples, like the GDP attribute value of XX city in 2023 being 520 billion yuan. A vector database supports semantic similarity retrieval, enabling the location of relevant facts even with different expressions. An inverted index enables rapid keyword location, with millisecond-level response times for query requests. Based on the article and topic in the text task instructions, relevant source facts are filtered. For the economic review writing task, the review agent extracts all 2023 economic indicator data from the pre-built knowledge index, including total GDP, growth rate, and industrial structure. This collection of authoritative facts constitutes the source set, representing verified, authentic data.

[0066] In some specific embodiments, a pre-defined knowledge index can be constructed in the following ways: Obtain multiple source documents for writing. The type of each source document corresponds to the corresponding writing task. Among them, source documents include, but are not limited to, industry review articles, statistical data, institutional directories, and annual basic materials. Each source document is parsed, segmented into sentences and paragraphs, and cleaned in a structured manner to remove invalid and redundant content and retain the valid main text information; Extract core basic elements from the cleaned source documents. Core basic elements include, but are not limited to, entity types: time entities, location entities, and organization entities; and numerical types: absolute values, relative ratios, and structural proportions. The extraction methods include, but are not limited to, rule template matching extraction and pre-trained semantic model extraction. All standard entities and standard values ​​extracted from the source documents are organized and aggregated into a structured source set S, which serves as the sole benchmark for determining the authenticity of facts. The source set S is indexed and encapsulated to obtain a preset knowledge index, which includes, but is not limited to, vector database index, knowledge graph index, and inverted index.

[0067] Step S1033: Calculate the difference between the generated set and the source set. If the difference is not empty, the factual consistency check of the candidate text is determined to have failed.

[0068] Specifically, the difference operation can be understood as subtracting the source set from the generated set. The result includes all fact units that exist in the generated set but cannot be found in the source set. If the difference set is empty, it means that every fact in the candidate text can be found in the source set, and the fact consistency check passes. If the difference set is not empty, it means that there are fabricated or incorrectly referenced facts, and the fact consistency check fails.

[0069] Specific example: Suppose the candidate text states that the GDP in 2023 was 550 billion yuan, while the source set records 520 billion yuan. The generated set contains the fact unit GDP of 550 billion yuan, and the source set contains the fact unit GDP of 520 billion yuan. The two values ​​are different, the difference set is not empty, and the verification fails, triggering a circuit breaker. Now suppose the candidate text states that the tertiary industry accounts for 50%, and the source set also records 50%. The fact units match perfectly, the difference set is empty, and this fact passes the verification.

[0070] In some specific embodiments, the method further includes: Based on the domain knowledge corresponding to the text task instruction, a logical check is performed on the first value in the generated set. If the logical check fails, it is determined that the factual consistency check of the candidate text has failed.

[0071] In the embodiments of this application, logical verification includes, but is not limited to, mathematical identity verification, growth rate derivation verification, ratio calculation verification, and cross-indicator correlation verification. For example, in mathematical identity verification, taking the composition of GDP as an example, the sum of the added value of the primary industry, the added value of the secondary industry, and the added value of the tertiary industry must equal the total GDP. If the candidate text claims that the three industries are 26 billion, 234 billion, and 260 billion respectively, while the total GDP is 520 billion, the calculation shows that 26 billion plus 234 billion plus 260 billion equals 520 billion, and the verification passes. If the calculation result is 51 billion or 53 billion, there is a difference of 10 billion, and the logical verification fails. Another example is ratio calculation verification: the urbanization rate must equal the urban population divided by the resident population. If the candidate text claims that the urban population is 7 million, the rural population is 3 million, and the urbanization rate is 70%, the calculation shows that 7 million divided by 1000 equals 70%, and the verification passes. If the claim is that the urbanization rate is 75%, the calculated value does not match the actual value, and the logical verification fails.

[0072] Step S104: If the verification is successful, the candidate text is scored in multiple dimensions to obtain the final positive reward value of the candidate text, and the text status of the candidate text is marked as passed.

[0073] In some specific embodiments, step S104 includes steps S1041-S1044: Step S1041: Assign a basic positive reward value to the candidate text that passes both the format standardization check and the factual consistency check.

[0074] Specifically, this step awards a minimum reward to candidate texts that pass the first two rigorous reviews (format conformity verification and factual consistency check). The function of the basic positive reward value is to clearly distinguish the treatment boundary between those who pass and those who fail. Receiving this reward indicates that the candidate text has reached the passing grade for official document writing, which has the highest requirements for content accuracy and format conformity. It is formatted correctly and factually accurate, and is qualified to enter the refined quality assessment.

[0075] Step S1042: Calculate the scores of the candidate texts on multiple dimensions.

[0076] Specifically, such as Figure 3 As shown: In the multi-dimensional scoring process, evaluation models or evaluation functions can be called to score candidate texts from multiple dimensions such as logic, objectivity, and conciseness.

[0077] In some specific embodiments, logical evaluation includes, but is not limited to, detecting whether candidate text contains data-supported causal statements, whether year-on-year and month-on-month calculations are correct, whether key elements are fully covered, and whether inferences are appropriate rather than over-interpreted.

[0078] In some specific embodiments, the objective assessment includes, but is not limited to, scanning the density of subjective praise words, vague limiting words and intensifying adverbs, and calculating the ratio of factual statements to evaluative content.

[0079] In some specific embodiments, the refinement assessment includes, but is not limited to, identifying redundant expressions and cliché patterns, analyzing sentence efficiency, and the frequency of function word usage.

[0080] Step S1043: The scores on the multiple dimensions are weighted and summed according to the preset weights corresponding to each dimension to obtain the weighted sum result.

[0081] Specifically, a preset weight is set for each dimension in advance, for example: logic accounts for 25%, objectivity accounts for 20%, conciseness accounts for 20%, coherence accounts for 15%, and professionalism accounts for 20%. The number of dimensions and the preset weight for each dimension can be set according to the actual situation, and no specific limit is made here.

[0082] Step S1044: The weighted summation result is superimposed with the basic positive reward value to generate the final positive reward value of the candidate text.

[0083] In the embodiments of this application, such as Figure 4 As shown: Candidate A received a positive reward of "+8.5" after passing the format and specification gate, the factual consistency gate, and the refined scoring of semantic and style optimization, and its text status was "PASSED". Candidate B failed the format and specification gate, resulting in a circuit breaker that prevented further review steps from being performed, and thus received a negative reward of "-10", with its text status being "LEVEL_1". This demonstrates that Candidate A, by passing all gates sequentially, received a positive reward, making full use of computational resources; while Candidate B, by failing at the first level, had its subsequent evaluation terminated, thus achieving a computational efficiency benefit.

[0084] Step S105: Based on the text state and reward value of the multiple candidate texts, optimize the strategy model to be optimized to obtain the final model.

[0085] In some specific embodiments, step S105 includes steps S1051-S1054: Step S1051: Calculate the average reward of the multiple candidate texts.

[0086] Specifically, the average reward for multiple candidate texts can be calculated using the following formula:

[0087] in, This indicates the total number of candidate texts. Indicates the first The reward value for each candidate text. This represents the average reward.

[0088] Step S1052: For any candidate text among the plurality of candidate texts, calculate the relative advantage value of the candidate text based on the reward value of the candidate text and the average reward value.

[0089] Specifically, the relative advantage value of the candidate texts is calculated using the following formula:

[0090] in, Indicates the first The relative advantage value of each candidate text. Indicates the first The reward value for each candidate text. This represents the average reward, when A value greater than 0 indicates that the model's generation probability is better than average, and the model should increase its generation probability accordingly; conversely, when... A value less than 0 indicates that the model is inferior to the average level, and the probability of generating that model should be reduced.

[0091] Step S1053: Construct a policy gradient objective function based on the relative advantage values ​​of the multiple candidate texts.

[0092] Specifically, the policy gradient objective function is used to increase the generation probability of candidate texts with a positive relative advantage value, while decreasing the generation probability of candidate texts with a negative relative advantage value.

[0093] For example: Candidate Text A: Formatting Circuit Breaker → Reward: -10 (strongly negative); Candidate Text B: Factual Circuit Breaker → Reward: -20 (extremely negative, veto level); Candidate Text C: All passed → Reward: +5 (positive incentive); Relative rewards: A: -0.3, B: -0.6, C: +3.6; Relative rewards enable the model to explicitly learn: - Formatting errors (A) and factual errors (B) are both serious problems; - Factual errors (B) are more serious than format errors (A) and should never be tolerated; - Only by passing all (C) steps can you receive a positive reward.

[0094] This reward mechanism enables the optimized model to achieve the following technical effects: 1. Unqualified candidate texts are eliminated in the early stages (i.e., format conformity verification and factual consistency check), saving 70%+ computing power; 2. The model clearly knows "at which level the error occurred" and "how serious it is"; 3. Fabricating facts = extremely severe punishment, eliminating the possibility of models subjectively generating false facts and getting away with it from a systemic perspective.

[0095] In this embodiment, the review agent returns the calculated final reward R_total and status to the GRPO training framework. For candidate texts that fail to pass the circuit breaker, it receives a large negative score; for those that pass, it receives a positive score. The GRPO training framework utilizes these highly differentiated and clearly signaled reward scores generated by the architecture of this invention to efficiently guide model learning: prioritizing the avoidance of formatting and factual errors, and progressively optimizing the writing quality of the content, thereby achieving fast, stable, and reliable model optimization.

[0096] In this embodiment, after the model generates a candidate text, the system sends it to a strictly sequential, multi-level "gating" verification process. First, the output enters the "format and specification gating," where the system automatically checks whether it conforms to the preset document writing structure, item title format, number usage specifications, and whether there are any unofficial abbreviations or expressions. Any failure in format or specification will immediately trigger a "circuit breaker," halting all subsequent evaluations and awarding a significant negative score. The "survivors" who pass the first hurdle enter the "fact consistency gating," where the system rigorously verifies its consistency with the source material using procedural methods such as set difference and logical verification. Any discovery of "illusions" will also trigger a circuit breaker and be subject to the most severe penalty. Only candidate outputs that perfectly pass all circuit breaker gatings are eligible to enter the final multi-dimensional scoring and optimization phase, where advanced dimensions such as logic, objectivity, and conciseness are refined. Through this process, this invention achieves a transformation of the reward calculation process from a "black box" to a "white box," integrating the verification, penalty, and optimization stages for efficient management. The three-pronged approach of hard constraints (format, facts), clear and strong circuit breaker penalties, and soft preferences (style, writing style) optimization enables the model to quickly and efficiently learn to first comply with the "high-voltage line" rules and then gradually improve the quality of content, bringing decisive value benefits to the application of large models in serious and professional fields.

[0097] Step S1054: Perform multiple rounds of optimization on the policy model to be optimized according to the policy gradient objective function to obtain the final model.

[0098] The embodiments of this application have the following technical effects: 1. By constructing a hierarchical and cascading "circuit breaker" reward architecture (i.e., first performing format standardization checks on candidate texts, then performing factual consistency checks after passing the checks, and finally performing multi-dimensional scoring), the reward calculation is transformed from a parallel "one-size-fits-all" evaluation into a serial verification process with checkpoints. This solves the problem of wasted computing resources caused by ineffective in-depth evaluation of unqualified outputs and provides the model with a clear, priority-based learning path.

[0099] 2. By introducing a "fact consistency gating" (i.e., performing fact consistency checks on the candidate text) based on rule-based and procedural methods such as set difference and logical verification, the model-generated content is subjected to "zero tolerance" illusion detection, thereby solving the technical problem that the existing reward mechanism cannot prioritize "fact accuracy" and thus leads to insufficient content reliability.

[0100] 3. By deeply integrating the clear and strong negative reward signal generated by the circuit breaker mechanism with the GRPO algorithm, a clear and black-and-white training guide is provided for the policy model (i.e., the final model is obtained by optimizing the policy model based on the text state and reward value of the multiple candidate texts), thereby solving the problem of fuzzy traditional mixed reward signals, which leads to slow model convergence and unclear training direction.

[0101] Corresponding to the above implementation of the large model optimization method based on hierarchical cascaded circuit breaker rewards, this application also provides a large model optimization apparatus based on hierarchical cascaded circuit breaker rewards, used to execute the large model optimization method based on hierarchical cascaded circuit breaker rewards described in any of the above embodiments. Figure 5 As shown, the large model optimization device based on tiered cascaded circuit breaker rewards includes: The candidate text acquisition module is used to acquire multiple candidate texts; the multiple candidate texts are output by the strategy model to be optimized based on the text task instructions and the original text. The format standardization verification module is used to perform format standardization verification on any candidate text among the plurality of candidate texts. If the verification fails, a circuit breaker is triggered, a first negative reward value is assigned to the candidate text, and the text status of the candidate text is marked as the first circuit breaker state. The fact consistency verification module is used to perform fact consistency verification on the candidate text if the verification is successful, and to trigger a circuit breaker if the verification fails, assign a second negative reward value to the candidate text, and mark the text status of the candidate text as a second circuit breaker state; the absolute value of the second negative reward value is greater than the first negative reward value; The multi-dimensional scoring module is used to score the candidate text in multiple dimensions if the verification is successful, obtain the final positive reward value of the candidate text, and mark the text status of the candidate text as passed. The model optimization module is used to optimize the strategy model to be optimized based on the text state and reward value of the multiple candidate texts to obtain the final model.

[0102] Optionally, the format conformity verification module is further configured to perform structured parsing on the candidate text to obtain multiple structured fields, and verify whether the multiple structured fields conform to the preset hierarchical style specifications; the multiple structured fields include the title, entry title, body text, and data source; template matching is performed on the key format and content features of the candidate text using multiple regular expressions; the multiple regular expressions include a first regular expression for verifying the format of the entry title, a second regular expression for verifying the conformity of the use of numbers, and a third regular expression for verifying the accuracy of the full name of the organization; if the multiple structured fields do not conform to the preset hierarchical style specifications, or if the template matching of any regular expression fails, then the candidate text verification is determined to have failed.

[0103] Optionally, the fact consistency verification module is further configured to extract a first entity and a first value from the candidate text, and form a generated set based on the first entity and the first value; filter a second entity and a second value that match the text task instruction from a preset knowledge index, and form a source set based on the second entity and the second value; calculate the difference between the generated set and the source set, and if the difference is not empty, determine that the fact consistency verification of the candidate text has failed.

[0104] Optionally, the fact consistency verification module is further configured to perform logical verification on the first value in the generated set based on the domain knowledge corresponding to the text task instruction, and if the logical verification fails, it is determined that the fact consistency verification of the candidate text has failed.

[0105] Optionally, the multi-dimensional scoring module is also used to assign basic positive reward values ​​to candidate texts that simultaneously pass the format standardization check and the factual consistency check; calculate the scores of the candidate texts on multiple dimensions; the multiple dimensions include logical dimension, objectivity dimension, and conciseness dimension; perform weighted summation on the scores of the multiple dimensions according to the preset weights corresponding to each dimension to obtain a weighted summation result; and superimpose the weighted summation result with the basic positive reward value to generate the final positive reward value of the candidate text.

[0106] Optionally, the model optimization module is further configured to calculate the average reward of the plurality of candidate texts; for any candidate text among the plurality of candidate texts, calculate the relative advantage value of the candidate text based on the reward value of the candidate text and the average reward; the relative advantage value is positive, indicating that the text generation level of the candidate text is higher than the average level; the relative advantage value is negative, indicating that the text generation level of the candidate text is lower than the average level; construct a policy gradient objective function based on the relative advantage values ​​of the plurality of candidate texts; the policy gradient objective function is used to increase the generation probability of candidate texts with positive relative advantage values, while decreasing the generation probability of candidate texts with negative relative advantage values; and perform multiple rounds of optimization on the policy model to be optimized based on the policy gradient objective function to obtain the final model.

[0107] The large model optimization device based on hierarchical cascaded circuit breaker reward provided in the above embodiments of this application and the large model optimization method based on hierarchical cascaded circuit breaker reward provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by their stored applications.

[0108] This application also provides a computer device for executing the above-described large model optimization method based on tiered cascading circuit breaker rewards. Please refer to... Figure 6 This illustrates a schematic diagram of a computer device provided by some embodiments of this application. For example... Figure 6 As shown, the computer device 6 includes: a processor 600, a memory 601, a bus 602, and a communication interface 603. The processor 600, the communication interface 603, and the memory 601 are connected via the bus 602. The memory 601 stores a computer program that can run on the processor 600. When the processor 600 runs the computer program, it executes the large model optimization method based on hierarchical cascaded circuit breaker rewards provided in any of the foregoing embodiments of this application.

[0109] The memory 601 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 603 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0110] Bus 602 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Memory 601 is used to store programs. After receiving an execution instruction, processor 600 executes the program. The large model optimization method based on hierarchical cascaded circuit breaker rewards disclosed in any of the foregoing embodiments can be applied to processor 600, or implemented by processor 600.

[0111] The processor 600 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 600 or by instructions in software form. The processor 600 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 601. Processor 600 reads the information in memory 601 and, in conjunction with its hardware, completes the steps of the above method.

[0112] The computer device provided in this application embodiment and the large model optimization method based on hierarchical cascaded circuit breaker rewards provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0113] This application also provides a computer-readable storage medium corresponding to the large model optimization method based on hierarchical cascaded circuit breaker rewards provided in the foregoing embodiments. Please refer to... Figure 7 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the large model optimization method based on hierarchical cascaded circuit breaker rewards provided in any of the foregoing embodiments.

[0114] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0115] The computer-readable storage medium provided in the above embodiments of this application and the large model optimization method based on hierarchical cascaded circuit breaker rewards provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0116] It should be noted that: Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0117] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0118] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0119] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A large-scale model optimization method based on tiered cascading circuit breaker rewards, characterized in that, The method includes: Obtain multiple candidate texts; the multiple candidate texts are output by the strategy model to be optimized based on the text task instructions and the original text; For any candidate text among the plurality of candidate texts, perform a format standardization check on the candidate text. If the check fails, trigger a circuit breaker, assign a first negative reward value to the candidate text, and mark the text status of the candidate text as the first circuit breaker state. If the verification is successful, the candidate text is checked for factual consistency. If the verification fails, a circuit breaker is triggered, a second negative reward value is assigned to the candidate text, and the text status of the candidate text is marked as the second circuit breaker state. The absolute value of the second negative reward value is greater than the first negative reward value. If the verification is successful, the candidate text is scored in multiple dimensions to obtain the final positive reward value of the candidate text, and the text status of the candidate text is marked as passed. Based on the text states and reward values ​​of the multiple candidate texts, the model to be optimized is optimized to obtain the final model.

2. The method according to claim 1, characterized in that, Perform format compliance checks on the candidate text, including: The candidate text is parsed in a structured manner to obtain multiple structured fields, and the multiple structured fields are verified to conform to the preset hierarchical style specifications; the multiple structured fields include article title, entry title, body text, and data source; The candidate text is template matched using multiple regular expressions to determine its key format and content features. These regular expressions include a first regular expression for validating the format of the entry title, a second regular expression for validating the standardization of number usage, and a third regular expression for validating the accuracy of the full name of the organization. If the multiple structured fields do not conform to the preset hierarchical style specification, or if the template matching of any regular expression fails, then the candidate text validation is determined to have failed.

3. The method according to claim 1 or 2, characterized in that, Perform fact consistency checks on the candidate texts, including: Extract a first entity and a first value from the candidate text, and generate a set based on the first entity and the first value; Filter the second entity and the second value that match the text task instruction from the preset knowledge index, and form a source set based on the second entity and the second value; Calculate the difference between the generated set and the source set. If the difference is not empty, the factual consistency check of the candidate text is determined to have failed.

4. The method according to claim 3, characterized in that, The method further includes: Based on the domain knowledge corresponding to the text task instruction, a logical check is performed on the first value in the generated set. If the logical check fails, it is determined that the factual consistency check of the candidate text has failed.

5. The method according to claim 1 or 2, characterized in that, The candidate texts are scored across multiple dimensions to obtain their final positive reward value, including: A basic positive reward value is assigned to candidate texts that pass both the format conformity check and the factual consistency check. The candidate texts are scored on multiple dimensions, including logicality, objectivity, and conciseness. The scores on the multiple dimensions are weighted and summed according to the preset weights corresponding to each dimension to obtain the weighted sum result; The weighted summation result is superimposed with the basic positive reward value to generate the final positive reward value of the candidate text.

6. The method according to claim 1 or 2, characterized in that, Based on the text states and reward values ​​of the multiple candidate texts, the model to be optimized is optimized to obtain the final model, including: Calculate the average reward for the multiple candidate texts; For any candidate text among the plurality of candidate texts, a relative advantage value is calculated based on the reward value of the candidate text and the average reward value; a positive relative advantage value indicates that the text generation level of the candidate text is higher than the average level; a negative relative advantage value indicates that the text generation level of the candidate text is lower than the average level. A policy gradient objective function is constructed based on the relative advantage values ​​of the multiple candidate texts; the policy gradient objective function is used to increase the generation probability of candidate texts with positive relative advantage values, while decreasing the generation probability of candidate texts with negative relative advantage values. The final model is obtained by performing multiple rounds of optimization on the policy model to be optimized based on the policy gradient objective function.

7. The method according to claim 1 or 2, characterized in that, The text task instruction is used to instruct the strategy model to be optimized to generate the target document based on the original text.

8. A large-scale model optimization device based on hierarchical cascaded circuit breaker rewards, characterized in that, The device includes: The candidate text acquisition module is used to acquire multiple candidate texts; the multiple candidate texts are output by the strategy model to be optimized based on the text task instructions and the original text. The format standardization verification module is used to perform format standardization verification on any candidate text among the plurality of candidate texts. If the verification fails, a circuit breaker is triggered, a first negative reward value is assigned to the candidate text, and the text status of the candidate text is marked as the first circuit breaker state. The fact consistency verification module is used to perform fact consistency verification on the candidate text if the verification is successful, and to trigger a circuit breaker if the verification fails, assign a second negative reward value to the candidate text, and mark the text status of the candidate text as a second circuit breaker state; the absolute value of the second negative reward value is greater than the first negative reward value; The multi-dimensional scoring module is used to score the candidate text in multiple dimensions if the verification is successful, obtain the final positive reward value of the candidate text, and mark the text status of the candidate text as passed. The model optimization module is used to optimize the strategy model to be optimized based on the text state and reward value of the multiple candidate texts to obtain the final model.

9. A computer device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the large model optimization method based on hierarchical cascaded circuit breaker rewards as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the large model optimization method based on hierarchical cascaded circuit breaker rewards as described in any one of claims 1 to 7.

Images