Large model generation content processing methods and apparatus, electronic devices and computer media
By generating a short label mapping table and an evidence allowance list, the output of the large language model is constrained and its claim source short labels and evidence identifiers are parsed. This solves the problem of uncertainty in the content generated by the large language model, realizes the controllability of the source of the generated content and the traceability of the results, and improves the accuracy and consistency of the generated content.
Patent Information
- Application Number
- CN202611537602.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-30
- Publication Date
- 2026-11-17
AI Technical Summary
The uncertainty of the content generated by the large language model leads to unstable format of the generated results, missing fields, unclear references, and inconsistencies between the output content and the input context. This makes it difficult to verify the source and associate the version, affecting the interpretability and controllability of the generated content.
By generating a short tag mapping table and an evidence allowable list, the output of the large language model is constrained and its claim source short tags and evidence identifiers are parsed. Structured decoding and whitelist verification are then performed to ensure that the source of the generated content is controllable and the results are traceable.
This achieves controllability of the source of generated content and traceability of results, reduces the risk of uncontrollable model output, and improves the accuracy and consistency of generated content.
Smart Images

Figure CN123115356A_ABST
Abstract
Description
Technical Field
[0001] This disclosure belongs to the field of artificial intelligence technology, specifically relating to the fields of natural language processing, educational technology, large language models, etc., and in particular to a method and apparatus for processing content generated by a large model, an electronic device, and a computer-readable storage medium. Background Technology
[0002] With the development of large language model technology, content generation systems based on large language models have been widely used in scenarios such as intelligent education, intelligent customer service, content creation, and knowledge-based question answering. These systems typically receive user input text, assemble the user input, system prompts, historical context, and business rules into a model request, and then call the large language model to generate the corresponding text or structured data.
[0003] In practical applications, the content generated by large language models exhibits significant uncertainty. Because model output is based on probability sampling, the generated results may suffer from unstable formatting, missing fields, unclear citations, and inconsistencies between the output content and the input context. This is particularly problematic in complex tasks involving multiple rounds of interaction, multiple data sources, or multiple generation targets, where semantic conflicts or structural splicing errors may exist between the content returned by different model calls. Furthermore, when the system needs to review, trace, or incrementally update the generated content, it is often difficult to determine the specific input source, context version, and model call parameters upon which a particular piece of generated content was based, leading to insufficient interpretability and controllability of the generated results.
[0004] Therefore, in scenarios where large language models are used for structured content generation, there is a need for a technical solution that can automatically parse the model output, verify its source, and associate it with versions, thereby improving the accuracy, consistency, and traceability of the generated content. Summary of the Invention
[0005] This disclosure provides a method and apparatus for processing large model generation content, an electronic device, and a computer-readable storage medium.
[0006] According to the first aspect, a method for processing content generated by a large model is provided. This method includes: obtaining the compilation unit to be executed and the set of source nodes actually visible to the compilation unit in the current model call of the target large language model; generating a unique short label mapping table and an evidence allowance list based on the source node set; the short label mapping table is used to record the correspondence between source node identifiers and source short labels; the evidence allowance list includes the evidence identifiers corresponding to the evidence nodes in the source node set that are allowed to be referenced by the current compilation unit; according to the short label mapping table, attaching corresponding source short labels to the source node input representations of the current model call, and configuring the input representations with source short labels and the evidence allowance list in the request sent to the target large language model; and configuring structured output constraints including multiple output items in the request so that the model response returned by the target large language model carries the declaration of source short labels. The system generates labels and assertion evidence identifiers, where each output item includes a content field, an array of assertion source short labels, and an array of assertion evidence identifiers. It receives the model response returned by the target large language model, parses each output item in the model response, and generates content atoms and their atomic information. It obtains the assertion source short labels and assertion evidence identifiers for each content atom. When the assertion source short label is decoded through the short label mapping table, the source node identifier corresponding to the assertion source short label is written into the valid source field included in the content atom. When the assertion evidence identifier belongs to the evidence allowed list, the assertion evidence identifier is written into the valid evidence field included in the content atom. The system then persistently stores the content atoms that have completed verification processing and their verification status, bound to the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to this model call.
[0007] According to the second aspect, a large model generation content processing apparatus is provided, comprising: an acquisition unit configured to acquire a compilation unit to be executed and a set of source nodes actually visible to the compilation unit in the current model call of the target large language model; a generation unit configured to generate a unique short tag mapping table and an evidence permission list based on the source node set; the short tag mapping table is used to record the correspondence between source node identifiers and source short tags; the evidence permission list includes evidence identifiers corresponding to evidence nodes in the source node set that are allowed to be referenced by the current compilation unit; and a configuration unit configured to attach corresponding source short tags to the source node input representations of the current model call according to the short tag mapping table, and configure the input representations with source short tags and the evidence permission list into the request sent to the target large language model; and configure structured output constraints including multiple output items in the request so that the model response returned by the target large language model carries the declaration of source short tags. The system comprises: a tag and a declaration evidence identifier, wherein each output item includes a content field, an array of declaration source short tags, and an array of declaration evidence identifiers; a receiving unit configured to receive the model response returned by the target large language model, parse each output item in the model response, and generate content atoms and atomic information for each content atom; a verification unit configured to obtain the declaration source short tags and declaration evidence identifiers for each content atom; when the declaration source short tags are decoded through the short tag mapping table, the source node identifier corresponding to the declaration source short tags is written into the valid source field included in the content atom; when the declaration evidence identifier belongs to the evidence allowed list, the declaration evidence identifier is written into the valid evidence field included in the content atom; and a tracing unit configured to bind the content atoms that have completed verification processing and their verification status with the graph version, compilation unit identifier, and runtime identifier bound to the compilation unit corresponding to this model call, and then persistently store them.
[0008] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method as described in any implementation of the first aspect.
[0009] According to a fourth aspect, a computer-readable storage medium is provided having computer instructions stored thereon for causing a computer to perform the method as described in any implementation of the first aspect.
[0010] The large model generation content processing method and apparatus provided in the embodiments of this disclosure firstly obtains the compilation unit to be executed and the set of source nodes actually visible to the compilation unit in the current model call of the target large language model; secondly, based on the set of source nodes, a unique short tag mapping table and an evidence allowable list are generated; the short tag mapping table is used to record the correspondence between source node identifiers and source short tags; the evidence allowable list includes the evidence identifiers corresponding to the evidence nodes in the source node set that are allowed to be referenced by the current compilation unit; thirdly, according to the short tag mapping table, corresponding source short tags are attached to the source node input representations of the current model call, and the input representations with source short tags and the evidence allowable list are configured in the request sent to the target large language model; and a structured output constraint including multiple output items is configured in the request so that the model response returned by the target large language model carries the declaration source short tags. The system generates content atoms and declaration evidence identifiers, with each output item including a content field, an array of declaration source short tags, and an array of declaration evidence identifiers. Then, it receives the model response from the target large language model, parses each output item in the model response, and generates content atoms and their atomic information. Next, it obtains the declaration source short tags and declaration evidence identifiers for each content atom. When the declaration source short tags are decoded through a short tag mapping table, the source node identifier corresponding to the declaration source short tag is written into the valid source field included in the content atom. When the declaration evidence identifier belongs to the evidence allowed list, the declaration evidence identifier is written into the valid evidence field included in the content atom. Finally, the content atoms that have completed the verification process and their verification status are bound to the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to this model call and then persistently stored. Therefore, before the model call, the actual set of source nodes entering the context can be encoded into a short tag mapping table and an evidence allowed list, and structured output constraints can be used to require the model to declare the referenced source short tags and evidence identifiers when generating content. Because short tags are only valid within the current compilation unit, the model cannot generate seemingly reasonable source identifiers that are not in the current context by guessing or fabricating, thus reducing the risk of uncontrollable model output. Upon receiving the model response, the system performs decoding verification on the declared source short tags and whitelist verification on the declared evidence identifiers. Only identifiers that pass verification are written into the valid fields of the content atoms; content atoms that fail verification are marked as abnormal and await further review. Furthermore, the verified content atoms are persistently stored after being bound to the graph version, compilation unit identifier, and runtime identifier. This allows each content atom to be traced back to its graph state, compilation scope, and model call context at the time of its generation, thus meeting the technical requirements for source verification, version association, and traceability processing of generated content.
[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:
[0013] Figure 1 This is a flowchart of an embodiment of the content processing method for generating large models according to this disclosure;
[0014] Figure 2 This is a schematic diagram of a structure of an embodiment of the large model-generated content processing apparatus according to the present disclosure;
[0015] Figure 3 This is a block diagram of an electronic device used to implement the large model generation content processing method of the embodiments of this disclosure. Detailed Implementation
[0016] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.
[0017] The large model content generation method provided in this disclosure can be applied to various scenarios of generating structured content using large language models, such as intelligent educational courseware generation, intelligent customer service response generation, report document generation, and knowledge question and answer content generation. In these scenarios, the system typically needs to provide the large language model with a context containing multiple sources, multiple business fields, or multiple constraints, and requires the model to output content according to a predetermined structure. Due to the uncertainty of the model output, the generated content may have problems such as format deviations, improper source citations, or out-of-bounds evidence citations. Therefore, it is necessary to perform structured parsing, source verification, and version association on the model output to ensure that the generated content can be reviewed, traced, and reused subsequently.
[0018] In some processing methods, the system directly sends complete graph node identifiers or business object identifiers to the model and requires the model to reference these identifiers in its output. In this approach, node identifiers are typically composed of long strings or globally unique identifiers, occupying a significant amount of lexical space within the model context. Furthermore, because the model's referencing of identifiers is unrestricted, the model may output identifiers that appear reasonable but do not actually exist in the current context, making it difficult to distinguish between genuine sources and fictional content created by the model.
[0019] In other processing methods, the system only saves the final text result after model generation, without performing structured source verification and evidence verification on the generated content. When it is necessary to review or trace a certain segment of generated content, it can only rely on manual comparison of the original prompt words and model output, which is inefficient and prone to missing abnormal references. At the same time, due to the lack of a mechanism to associate generated content with specific model calls and context versions, when upstream inputs change, the system has difficulty determining which generated content needs to be re-verified or updated.
[0020] To address the technical requirements in the aforementioned scenarios, this disclosure provides a method for processing content generated by a large model. This method generates a short tag mapping table and an evidence allowance list for the set of source nodes that actually enter the context before the model is invoked. It also requires the model to declare the source short tags and evidence identifiers it references when generating content through structured output constraints. After the model response is returned, the declared content is decoded and verified, and a whitelist is verified. Finally, the verified content atoms are bound to the graph version, compilation unit identifier, and runtime identifier and then persistently stored. This achieves controllable source of model-generated content, verifiable evidence, and traceable results.
[0021] Figure 1 A flow 100 is shown as an embodiment of the large model generation content processing method according to this disclosure, which includes the following steps:
[0022] Step 101: Obtain the compilation unit to be executed and the set of source nodes that the compilation unit is actually visible in this model call of the target large language model.
[0023] In this embodiment, a compilation unit refers to a set of tasks that can be independently submitted to the target large language model, extracted from the educational root cue word graph under the constraint of a "large model context budget". The compilation unit includes model-specific messages, context nodes, output modes, and runtime parameters. Through the compilation unit, infinitely long natural language cue words can be discretized into controlled, schedulable computational packages, avoiding context overflow.
[0024] In this embodiment, the source node set refers to the graph nodes (such as topics, objectives, core concepts, policy documents, etc.) that are actually selected and serialized into the large language model within the execution scope of the current compilation unit. The source node set clarifies the "legitimate context boundary" of the model generation and serves as the benchmark for subsequent anti-illusion verification.
[0025] In this embodiment, step 101 may include: obtaining graph data (or educational root prompt word graph) containing multiple source nodes and target output slots; determining the set of source nodes actually visible in this model call from the graph data based on the context budget of the target large language model, and constructing the corresponding compilation unit.
[0026] In this embodiment, step 101 includes: determining the output slot to be generated, and calculating the available context budget after deducting system protocol and output reservation in this model call based on the model profile of the target large language model; extracting the hard dependency closure of the output slot by reverse traversal based on the execution projection of the educational root prompt word graph, and adding the nodes in the hard dependency closure to the set of actually visible source nodes; calculating the ratio of the utility value of each non-hard dependency node to the lexical cost for the output slot for each non-hard dependency node in the graph; and, if the total lexical cost of the hard dependency closure does not exceed the available context budget, adding the non-hard dependency nodes to the set of actually visible source nodes in descending order of the ratio until the total lexical cost reaches the upper limit of the available context budget, thereby dynamically generating the compilation unit to be executed.
[0027] Optionally, step 101 above may further include: maintaining a directed acyclic graph of execution units consisting of multiple compilation units and their execution dependency edges, and monitoring the in-degree status of each compilation unit in real time; in response to detecting that the in-degree of the target compilation unit drops to zero, that is, all the preceding compilation units that the target compilation unit depends on have successfully generated the corresponding content atoms and obtained the verification pass status, moving the target compilation unit from the blocking queue to the ready queue; extracting the target compilation unit from the ready queue as the compilation unit to be executed; merging the course global shared node corresponding to the target compilation unit and the content atom summary generated by the preceding compilation units as the set of source nodes actually visible in this model call.
[0028] Optionally, step 101 may further include: in response to a change in the content of a target source node in the educational root prompt word graph, calculating the new content hash value of the target source node; based on the reverse dependency edges in the graph, finding all affected output slots that depend on the target source node, and marking the state of the historical compilation units corresponding to the affected output slots as invalid states; extracting the historical compilation units marked as invalid states as compilation units to be executed, and using the updated target source node and the unchanged associated dependency nodes as the set of source nodes actually visible in this model call, so as to trigger a local recompilation for the affected content.
[0029] Step 102: Based on the source node set, generate a unique short label mapping table and an evidence allowed list.
[0030] In this embodiment, the short label mapping table is used to record the correspondence between source node identifiers and source short labels. The short label mapping table is also a data structure that maps long string source node identifiers (such as "EV-POLICY-2026-001") to extremely short identifiers (such as "n01", "n02"). The short label mapping table significantly reduces the "output reserve" consumed by large models when outputting structured JSON due to repeated references to long IDs, while forcing the model to reference within a controlled label space. The evidence allowed list includes the evidence identifiers corresponding to the evidence nodes in the source node set that are allowed to be referenced by this compilation unit.
[0031] In this embodiment, the evidence allowance list is a pre-calculated set of legitimate evidence IDs for a specific output slot. Any evidence ID returned by the model that is not in this list is judged as a hallucination or illegal citation. This transforms the large model's "unexplainable black-box generation" into "deterministically verifiable white-box lookup," addressing the pain point of AI citing false policies or documents when generating educational content.
[0032] In an optional embodiment, generating a unique short tag mapping table based on the source node set includes: obtaining the segmenter configuration information from the model profile of the target large language model; based on the segmenter configuration information, selecting a set of character sequences whose encoding length in the target large language model is strictly equal to one lexical unit within a preset character space as a candidate tag pool; sorting the source nodes in the source node set in descending order according to their graph priority or expected frequency of citation; sequentially extracting word metacharacter sequences from the candidate tag pool and mapping them to the corresponding source nodes according to the sorting results, thereby generating a short tag mapping table; so that when the target large language model outputs the short tag array of the source declaration, the output lexical cost of each source reference is deterministically minimized to the word metalevel.
[0033] In another optional embodiment, a unique short tag mapping table is generated based on the source node set, including: parsing the node type (node_type) of each source node in the source node set in the educational root cue word graph; the node type includes at least policy basis, core concept, teaching case or learning objective; for each node type, a corresponding minimalist semantic prefix is pre-configured; the minimalist semantic prefix corresponding to each source node is concatenated with its local index in the current compilation unit to generate a semantic short tag with type hints, and recorded in the short tag mapping table; so that when the target large language model generates content atoms, it can accurately recall the context within the corresponding node type range based on the attention cues of the minimalist semantic prefix, reducing the illusion probability of cross-type misreferences.
[0034] In another optional embodiment, a unique short tag mapping table is generated based on the source node set, including: identifying the node scope in the source node set and dividing the source node set into a globally shared node subset and a locally specific node subset; for the globally shared node subset, based on its graph version and global node identifier, applying a deterministic global mapping algorithm to generate global short tags that are consistent across all compilation units; for the locally specific node subset, after excluding already occupied global short tags, dynamically allocating local short tags within the namespace of the current compilation unit; merging the global short tags and local short tags to form a short tag mapping table; so that when multiple compilation units are executed concurrently, the globally shared node subset and its associated global short tags form a byte-level stable shared prefix in the input sequence, thereby improving the hit rate of the prompt cache on the target large language model server.
[0035] In this embodiment, generating the evidence allowable list based on the source node set includes: checking the node type of each node in the source node set; filtering out nodes whose node type is evidence node; extracting the unique in-graph identifier ID of these filtered evidence nodes; and assembling all the extracted evidence node IDs into an array or a string of a specific format, which generates the evidence allowable list specific to this model call.
[0036] When generating prompts or system messages specific to large models, this list should be injected into the prompts. Simultaneously, the structured output constraints should explicitly require large models to "not output evidence IDs that do not appear in the evidence allowed list."
[0037] Step 103: According to the short label mapping table, attach the corresponding source short label to the source node input representation of this model call, and configure the input representation with source short label and the evidence allowed list into the request sent to the target large language model; and configure the structured output constraints including multiple output items in the request so that the model response returned by the target large language model carries the claim source short label and claim evidence identifier.
[0038] In this embodiment, each output item includes a content field, an array of short tags for claiming source, and an array of claiming evidence identifiers.
[0039] In an optional embodiment, the request is configured with structured output constraints including multiple output items, including: calculating the available output lexicon budget for this model call based on the model profile of the target large language model and the input consumption of this call; parsing the preset complete target output schema and extracting the empirical lexicon consumption weight of each output item; when the available output lexicon budget is less than the total expected consumption of the complete target output schema, performing dynamic pruning on the complete target output schema: marking non-hard-dependent output items as deferred generation, and / or dynamically attaching maximum item number limit parameters to the declaration source short label array and declaration evidence identifier array in the structured output constraints; configuring the dynamically pruned structured output constraints into the request sent to the target large language model to ensure that the model returns a response according to the pruned structure and avoids JSON structure truncation due to exceeding the output lexicon budget.
[0040] In another optional embodiment, the request is configured with structured output constraints including multiple output items, including: detecting whether the model profile of the target large language model supports tool invocation capability; if so, converting the content field, the short tag array of the claim source, and the array of claim evidence identifiers into parameter definitions of the virtual tool function; configuring the virtual tool function as an available tool in the request sent to the target large language model, and forcing the target large language model to call the virtual tool function in the response by setting a forced tool selection parameter; parsing each output item in the model response, including: intercepting the tool invocation request returned by the target large language model, and directly extracting the content atoms and corresponding atom information from the parameter dictionary of the tool invocation request without performing structured parsing on the body text of the model response.
[0041] In another optional embodiment, a structured output constraint including multiple output items is configured in the request, including: obtaining all successor dependency node identifiers of the compilation unit in the directed acyclic graph of the execution unit; in the structured output constraint, in addition to configuring the content field, the array of declaration source short tags, and the array of declaration evidence identifiers, a successor routing field constraint is dynamically injected, and the successor dependency node identifier is used as an optional enumeration value of the field; so that when the target large language model generates the current content atom, it outputs the routing identifiers passed to the successor dependency nodes in conjunction with the context; then, when the system parses the model response, it extracts the routing identifiers and dynamically adjusts the priority or in-degree state of the corresponding successor compilation unit in the directed acyclic graph of the execution unit according to the routing identifiers.
[0042] Step 104: Receive the model response returned by the target large language model, parse each output item in the model response, and generate content atoms and atomic information for each content atom.
[0043] In this embodiment, the atomic information includes the atomic identifier of the content atom and the output slot to which the content atom belongs.
[0044] In this embodiment, the content atom is the smallest addressable and reusable data object obtained after the large model generation result is structured and parsed. The content atom not only includes the generated text, but also encapsulates metadata such as the graph version, compilation unit ID, and real source node ID. By using content atoms, the traditional method of "saving the entire text" is abandoned, enabling the generated courseware content to have database-level traceability and granularity.
[0045] In this embodiment, the graph version is a globally unique identifier (such as a Merkle root) calculated from the content hashes of all nodes and edges in the educational root prompt word graph. The graph version enables "snapshotting" of the generated content, allowing the system to accurately identify which content atoms were generated based on the old graph version when upstream curriculum standards or policies change.
[0046] In this embodiment, after sending a request to the target large language model, the processing system waits for and receives the model response. The model response can be a complete structured object returned in a single instance, or multiple incremental data blocks returned in a streaming manner. For streaming responses, the processing system can either concatenate the incremental data blocks into a complete response after reception, or parse the received portions in real time during the reception process.
[0047] The processing system parses each output item in the model response, generating content atoms and their atomic information. A content atom can be the smallest addressable object in the generated result, with each content atom corresponding to an output item in the model response. Atom information can include the atom identifier of the content atom and the output slot to which it belongs. For example, a content atom can be assigned a globally unique atom identifier ATOM-M04-CONCEPT-0007 and its output slot can be marked as S02.M04.CONCEPT. The data structure of the content atom can also include fields directly provided by the model response, such as content fields, arrays of declared source short labels, and arrays of declared evidence identifiers, as well as valid source fields, valid evidence fields, and verification status fields filled in subsequent verification steps.
[0048] In some embodiments, when the target large language model does not support native structured output and uses a structured text template, the model response may exhibit format deviations, such as missing closing brackets in JSON fields, mismatched string quotes, or missing fence markers. The processing system retains the original model response before parsing and performs format correction on the response's structure markers, fences, or punctuation. For example, if a missing closing curly brace is detected at the end of a JSON object, it can be automatically added; if an unescaped newline character is detected in a string value, it can be replaced with an escape sequence. After format correction, the processing system performs structure validation on the corrected response to confirm that it conforms to the expected output pattern before extracting each output item to generate content atoms. If structure validation fails, the processing system can mark the original response as formatting abnormal and trigger manual review or automatic recompilation, rather than directly discarding the original response or overwriting the original data. This preserves the original model output for auditing while improving the success rate of structured parsing through format correction.
[0049] Step 105: Obtain the short label of the claim source and the claim evidence identifier of each content atom in the multiple content atoms; when the short label of the claim source is decoded through the short label mapping table, write the source node identifier corresponding to the short label of the claim source into the valid source field included in the content atom; when the claim evidence identifier belongs to the evidence allowed list, write the claim evidence identifier into the valid evidence field included in the content atom.
[0050] In this embodiment, the atomic information of the content atom includes at least the compilation scope node field, the declaration source field, and the verification source field. The above-mentioned writing the source node identifier corresponding to the declaration source short tag into the valid source field included in the content atom includes: writing the identifier of the actual visible source node set in this model call into the compilation scope node field; writing the source node identifier directly corresponding to the declaration source short tag parsed in the model response into the declaration source field; calculating the intersection of the declaration source field and the compilation scope node field, and writing the intersection result into the verification source field, thereby filtering out the fictitious illegitimate sources of the target large language model.
[0051] In this embodiment, the system reads the array of declared source short tags and the array of declared evidence identifiers included in each content atom. For each declared source short tag, the processing system queries the short tag mapping table to determine whether a corresponding entry exists in the table. If a corresponding entry exists, the short tag is mapped to the corresponding source node identifier, and the source node identifier is written into the valid source field of the content atom. For example, if the declared source short tag of the content atom is n01, and the corresponding source node identifier obtained after querying the short tag mapping table is R05, then R05 is written into the valid source field. If the declared source short tag cannot be decoded through the short tag mapping table, for example, if the model outputs n99 which is not in the mapping table, the processing system writes the short tag into the unparsed source field and prohibits it from being written into the valid source field.
[0052] For each declared evidence identifier, the processing system queries the evidence allowed list to determine if the evidence identifier belongs to the list. If it does, the evidence identifier is written into the valid evidence field of the content atom. For example, if the evidence identifier declared by the content atom is EV-POLICY-001, and this identifier exists in the evidence allowed list, it is written into the valid evidence field. If the declared evidence identifier does not belong to the evidence allowed list, for example, if the model outputs EV-FAKE-999, the processing system prohibits writing this evidence identifier into the valid evidence field and marks the entire content atom as requiring manual review, preventing it from automatically progressing to the verified status. This prevents evidence from being mistakenly identified as valid citations due to fictitious sources or illegal references in the model, achieving controlled processing of abnormal content through field isolation and status marking.
[0053] In this embodiment, when decoding fails, the writing of the short label of the claim source is intercepted and a source anomaly flag is triggered; when the claim evidence identifier does not exist in the evidence allowed list, the writing of the claim evidence identifier is intercepted and the corresponding content is atomically marked as requiring review.
[0054] Step 106: After binding the content atoms that have completed the verification process and their verification status with the graph version, compilation unit identifier and runtime identifier of the compilation unit corresponding to this model call, the data is persistently stored.
[0055] In this embodiment, the content atoms whose source and evidence verification was completed in step 106, along with their verification status, are bound to the context information of this model call and stored in persistent storage. The bound context information includes the graph version bound to the compilation unit, the compilation unit identifier, and the runtime identifier. The graph version can be the overall hash or version number corresponding to the current graph state; the compilation unit identifier can uniquely identify the compilation unit used in this model call; and the runtime identifier can uniquely identify the model call process. By binding the content atoms with this information, each content atom can be traced back to the graph state, compilation scope, and model call context at the time of its generation.
[0056] In an optional embodiment, the content atom that has completed the verification process and its verification status are persistently stored after being bound to the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to the current model call. This includes: after updating the verification status of the content atom to "verification passed" and writing it to the graph database or relational database, triggering a state update event of the directed acyclic graph of the execution unit; in response to the state update event, querying all subsequent compilation units that depend on the current compilation unit in the directed acyclic graph of the execution unit; decrementing the in-degree of each found subsequent compilation unit by one; and in response to detecting that the in-degree of any subsequent compilation unit has dropped to zero, updating the state of the subsequent compilation unit from blocked to ready and adding it to the scheduling queue to trigger the next model call based on the content atom as the upstream context dependency.
[0057] In another optional embodiment, the content atoms that have completed the verification process and their verification status are bound to the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to this model call and then persistently stored. This includes: querying the content atom database to see if there are any historical content atoms belonging to the historical graph version and whose verification status is verified as passed, based on the output slot identifier corresponding to the current compilation unit; if they exist, comparing the new graph version bound to the current compilation unit with the historical graph version; if it is confirmed that the new graph version is later than the historical graph version, updating the verification status of the historical content atom to obsolete or replaced, and retaining its original data for version auditing or rollback; persistently writing the content atoms generated by this model call with the new graph version, and marking the verification status of the content atoms as currently valid.
[0058] In another optional embodiment, the content atoms that have completed the verification process and their verification status are bound to the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to this model call and then persistently stored. This includes: obtaining the globally unique idempotent key generated during the compilation phase based on the graph version, target output slot, and target large language model profile; performing anti-duplicate conflict detection in the content atom database using the globally unique idempotent key before performing persistent storage; if the globally unique idempotent key already exists and the corresponding status is verified, the current persistent storage operation is intercepted, and the existing historical content atom is directly returned to avoid duplicate model billing and dirty data writing due to network timeout retries; if no conflict is detected, the globally unique idempotent key, content atom, graph version, compilation unit identifier, and runtime identifier are submitted to the content atom database as a single transaction.
[0059] The large model generation content processing method provided in the embodiments of this disclosure first obtains the compilation unit to be executed and the set of source nodes actually visible to the compilation unit in the current model call of the target large language model; second, based on the set of source nodes, a unique short tag mapping table and an evidence allowable list are generated; the short tag mapping table is used to record the correspondence between source node identifiers and source short tags; the evidence allowable list includes the evidence identifiers corresponding to the evidence nodes in the source node set that are allowed to be referenced by the current compilation unit; third, according to the short tag mapping table, the corresponding source short tags are attached to the source node input representations of the current model call, and the input representations with source short tags and the evidence allowable list are configured in the request sent to the target large language model; and the structured output constraints including multiple output items are configured in the request so that the model response returned by the target large language model carries the declaration source short tags and the statement. The system identifies the evidentiary identifiers, where each output item includes a content field, an array of short labels declaring the source, and an array of evidentiary identifiers. Then, it receives the model response from the target large language model, parses each output item in the model response, and generates content atoms and their atomic information. Next, it obtains the short labels declaring the source and evidentiary identifiers for each content atom. When the short labels declaring the source are decoded through the short label mapping table, the source node identifier corresponding to the short label is written into the valid source field of the content atom. When the evidentiary identifier belongs to the evidence allowed list, it is written into the valid evidentiary field of the content atom. Finally, the content atoms that have completed the verification process and their verification status are bound to the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to this model call and then persistently stored.
[0060] Optionally, the above-mentioned large model generation content processing method further includes: in response to a change in the content of the target source node in the educational root prompt word graph, calculating the new content hash value of the target source node; when the new content hash value is different from the old content hash value, traversing based on the reverse dependency edges in the graph to determine the set of content atoms bound to the target source node identifier and the corresponding affected compilation units; marking the state of the affected compilation units as invalid and revoking the verified state of their corresponding content atoms; re-executing the steps of obtaining the source node, generating the mapping table, model calling and verification processing only for the affected compilation units marked as invalid, and persistently storing the newly generated content atoms with the new graph version, while the unaffected content atoms retain their original verified state.
[0061] Optionally, after persistently storing the content atoms that have completed the verification process and their verification status, the large model generation content processing method further includes: in response to all content atoms corresponding to the compilation unit obtaining a verification status that has passed verification, reducing the in-degree of all subsequent compilation units that depend on the compilation unit in the directed acyclic graph of execution unit dependencies; when the in-degree of any subsequent compilation unit is detected to be reduced to zero, adding the subsequent compilation unit to the ready queue to trigger the model call and content processing flow for that subsequent compilation unit.
[0062] In some optional implementations of this disclosure, the above-mentioned content atom that has completed the verification process and its verification status are bound to the graph version, compilation unit identifier and runtime identifier of the compilation unit corresponding to this model call and then persistently stored, including: obtaining the original response storage address of the model response; binding the content atom, the valid source field, the valid evidence field, and the actual set of source nodes visible to the compilation unit corresponding to this model call in this model call, the graph version corresponding to the compilation unit, the compilation unit identifier, the runtime identifier and the original response storage address together, and storing them in the content atom database as traceability record information.
[0063] In this optional implementation, the original response storage address of the model response can be obtained, and the content atoms, valid source field, valid evidence field, and the actual set of source nodes visible to the compilation unit corresponding to this model call, the graph version corresponding to the compilation unit, the compilation unit identifier, the runtime identifier, and the original response storage address can be bound together and stored in the content atom database as traceability record information. The original response storage address can be a Uniform Resource Locator (URL) or path identifier pointing to the original model response file stored in object storage or the file system. This allows for direct retrieval of the original model output based on the original response storage address associated with the content atoms during subsequent auditing or anomaly investigation, and comparison with the verified content atoms, further improving the traceability and auditability of the generated content.
[0064] In this embodiment, each record in the content atom database can include fields such as the content atom's text, type, module, output slot, valid source field, valid evidence field, graph version, compilation unit identifier, runtime identifier, number of input tokens, number of output tokens, verification status, and creation timestamp. The processing system can also create indexes for the content atoms to enable fast queries based on output slot, graph version, and verification status. For example, when querying all verified concept resolution content atoms for a course under a specific graph version, the index can quickly locate the set of records that meet the criteria.
[0065] In some embodiments of this disclosure, the above-mentioned large model generation content processing method further includes: in response to the short tag of the declared source being unable to be decoded through the short tag mapping table, writing the short tag of the declared source into the unparsed source field and prohibiting the short tag of the declared source from being written into the valid source field; in response to the declared evidence identifier not belonging to the evidence allowed list, prohibiting the declared evidence identifier from being written into the valid evidence field, marking the entire content atom as requiring manual review, and prohibiting the content atom from automatically flowing to the verified state.
[0066] The large model generation content processing method provided by this optional implementation can prevent the misuse of evidence from fictitious sources or illegal citations in the model. It isolates abnormal content by using unparsed source fields and requiring manual review, thus preventing unverified generated content from directly entering the downstream process.
[0067] In some optional implementations of this disclosure, generating a unique short tag mapping table as described above includes: extracting the source node identifier of each source node in the source node set, generating an initial short tag based on a truncated hash or an auto-incrementing sequence; performing collision detection within the scope of the current compilation unit, and if a collision is detected, re-encoding by extending the tag characters or adding a disambiguation sequence number to ensure that the short tag is unique within the scope of the current compilation unit, and generating a short tag mapping table.
[0068] In this optional implementation, the above-mentioned generation of initial short tags based on truncated hash or auto-incrementing sequence includes: obtaining the segmenter profile corresponding to the target large language model; and selecting a character sequence whose encoding length in the target large language model does not exceed the preset number of tokens as the initial short tag according to the segmenter profile, so that the total number of output tokens occupied by the structured output constraints due to the inclusion of multiple claim source short tags is strictly less than the output reserved budget configured in the target large language model profile.
[0069] In this optional implementation, the source node identifiers of each source node in the source node set are extracted, and initial short labels are generated based on truncated hashes or auto-incrementing sequences. For example, source node identifier R05 can be mapped to short label n01, and source node identifier R10 can be mapped to short label n02. The truncated hash can be a string obtained by performing a hash operation on the source node identifier and taking the first few characters; the auto-incrementing sequence can be a sequence number string assigned sequentially according to the order in which the source nodes appear in the set. Since the initial short labels may collide, the processing system performs collision detection within the scope of the current compilation unit. If two different source nodes are detected to be mapped to the same short label, the label characters are extended or a disambiguation sequence number is added for re-encoding to ensure that the short label is unique within the scope of the current compilation unit. For example, if n01 has been assigned to R05, and the truncated hash of another node R06 also produces n01, the short label of R06 can be re-encoded as n01a or n01_2. After collision detection and re-encoding are completed, the processing system generates a short label mapping table and stores it in the runtime context of this model call.
[0070] This optional implementation provides a method for generating a short tag mapping table. By extending the tag characters or adding disambiguation numbers for re-encoding, it ensures the uniqueness of short tags within the current compilation unit and generates a short tag mapping table. This maintains tag conciseness while preventing different source nodes from being mapped to the same short tag, ensuring a unique correspondence during the decoding stage.
[0071] In some optional implementations of this disclosure, the above-mentioned configuration of structured output constraints including multiple output items in the request includes: configuring a structured response mode when native structured output is supported, based on the interface capabilities of the target large language model; configuring a structured text template with start and end markers in the prompt words when native structured output is not supported; parsing each output item in the model response, including: retaining the original model response, performing format repair on the structural markers, fences, or punctuation of the response, and performing structural verification and source and evidence identification verification on the repaired response.
[0072] In this optional implementation, the above-mentioned structural validation of the repaired response includes: attempting to use a deterministic JSON parser to perform punctuation and fence repair on the model response; if the repair still fails to pass the schema validation of the structured output constraints, the original model response is retained, and a low-temperature retry call that only allows format repair is initiated to the target large language model, taking the original model response and error message as input, and requiring the model to return each output item that conforms to the specification.
[0073] In this optional implementation, the processing system also configures structured output constraints, including multiple output items, in the request to ensure that the model response returned by the target large language model carries source tag declarations and evidence identifier declarations. Each output item includes a content field, an array of source tag declarations, and an array of evidence identifier declarations. Depending on the interface capabilities of the target large language model, when native structured output is supported, the processing system can configure a structured response mode, such as JSONSchema, where the output object explicitly defines an array of items, and each item contains content, source_tags, and evidence_ids fields. In models that do not support native structured output, the processing system can configure a structured text template with start and end markers in the prompt words, such as appending example output formats and fence markers to the end of the prompt words, guiding the model to return content according to a predetermined format. Through structured output constraints, each content fragment generated by the model can be associated with explicit source and evidence claims, providing a structured foundation for subsequent verification.
[0074] The optional implementation provides a method for configuring structured output constraints that can adapt to the structured output capabilities of different model interfaces. Even when there are slight deviations in the format returned by the model, the original response can be preserved and structured data can be restored through format repair, ensuring the input quality of subsequent verification steps.
[0075] In some optional implementations of this disclosure, obtaining the compilation unit to be executed and the set of source nodes actually visible to the compilation unit in this model call includes: receiving the compilation unit passed in by the pre-compiler; obtaining the list of graph nodes actually written into the context by the pre-compiler in this model call, and using the list of graph nodes as the set of source nodes actually visible.
[0076] In this optional implementation, the processing system can receive compilation units from the pre-compiler. A compilation unit can be the smallest scheduling unit submitted to the target large language model in a single model call. The compilation unit may also include information such as: target output slot identifier, selected source node set, model profile identifier, and output mode configuration.
[0077] A pre-compiler can be a module that, during the upstream execution phase, decomposes and schedules generation tasks based on the graph structure, context budget, and dependencies, and then passes the compilation results to the processing system. The processing system can obtain a list of graph nodes actually written into the context during this model call from the pre-compiler, and use this list of graph nodes as the actual set of visible source nodes. This allows for direct reuse of the node information determined during the pre-compilation phase, avoiding redundant calculations, while ensuring that the obtained set of source nodes is strictly consistent with the actual visible context of the model.
[0078] In this optional implementation, each source node in the graph node list can have a source node identifier, such as R05, R10, EV-POLICY-001, etc., to uniquely identify the node in the graph.
[0079] In some embodiments, the compilation unit may also carry a graph version identifier, which can be a content hash or version number corresponding to the current graph state. When an upstream node changes, the graph version identifier is updated accordingly, enabling subsequent processing to distinguish the generated content under different graph versions. After obtaining the compilation unit and the set of source nodes, the processing system can temporarily store this information in the runtime context of this model call for use in subsequent steps.
[0080] In some embodiments of this disclosure, the above-mentioned large model generation content processing method further includes: performing batch structure verification on multiple content atoms generated in the same model call of the current compilation unit and belonging to the same output slot, detecting whether all the declaration source short tags of each content atom have corresponding entries in the short tag mapping table, and whether all the declaration evidence identifiers of each content atom exist in the evidence allowed list; in response to detecting that the declaration source short tag of any content atom cannot be decoded or the declaration evidence identifier is invalid, marking the entire set of content atoms corresponding to the compilation unit or output slot as a verification failure state, prohibiting the set of content atoms from entering the downstream release process, and triggering manual review or automatic recompilation.
[0081] In this embodiment, the processing system can also perform batch structure verification on multiple content atoms generated in the same model call within the same compilation unit and belonging to the same output slot. Batch structure verification checks whether all short source declaration tags for each content atom have corresponding entries in the short tag mapping table, and whether all declaration evidence identifiers for each content atom exist in the evidence allowed list. In response to the detection that the short source declaration tag of any content atom cannot be decoded or that the declaration evidence identifier is invalid, the processing system marks the entire set of content atoms corresponding to the compilation unit or output slot as failing verification, prohibits the set of content atoms from entering the downstream release process, and triggers manual review or automatic recompilation. This avoids some abnormal atoms being missed and mixed into the verified set in output slots containing multiple content atoms, thereby achieving overall quality control at the output slot level. For example, if the output slot S02.M04.CONCEPT generates three content atoms, two of which pass source verification while the third declares an illegal evidence identifier, then all three atoms are marked as failing verification and will be processed after review or recompilation.
[0082] The large model content processing method provided in this embodiment can avoid missing anomalies in individual content atoms. It achieves overall quality control at the output slot level through batch verification, preventing some abnormal atoms from being mixed into the verified set.
[0083] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a large model generation content processing apparatus, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.
[0084] like Figure 2 As shown, the large model generation content processing device 200 provided in this embodiment includes: an acquisition unit 201, a generation unit 202, a configuration unit 203, a receiving unit 204, a verification unit 205, and a source tracing unit 206. The acquisition unit 201 can be configured to acquire the compilation unit to be executed and the set of source nodes actually visible to the compilation unit in the current model call of the target large language model. The generation unit 202 can be configured to generate a unique short tag mapping table and an evidence permission list based on the source node set; the short tag mapping table records the correspondence between source node identifiers and source short tags; the evidence permission list includes the evidence identifiers corresponding to the evidence nodes in the source node set that are allowed to be referenced by the current compilation unit. The configuration unit 203 can be configured to attach corresponding source short tags to the source node input representation of this model call according to the short tag mapping table, and configure the input representation with source short tags and the evidence allowed list into the request sent to the target large language model; and configure structured output constraints including multiple output items in the request so that the model response returned by the target large language model carries the claim source short tags and claim evidence identifiers, wherein each output item includes a content field, an array of claim source short tags, and an array of claim evidence identifiers. The receiving unit 204 can be configured to receive the model response returned by the target large language model, parse each output item in the model response, and generate content atoms and atomic information for each content atom. The verification unit 205 can be configured to obtain the claim source short tags and claim evidence identifiers of each content atom in the multiple content atoms; when the claim source short tags are decoded through the short tag mapping table, write the source node identifier corresponding to the claim source short tag into the valid source field included in the content atom; when the claim evidence identifier belongs to the evidence allowed list, write the claim evidence identifier into the valid evidence field included in the content atom. The aforementioned tracing unit 206 can be configured to persistently store the content atoms that have completed the verification process and their verification status, which are bound to the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to this model call.
[0085] In this embodiment, the specific processing and technical effects of the large model generation content processing device 200, including the acquisition unit 201, generation unit 202, configuration unit 203, receiving unit 204, verification unit 205, and tracing unit 206, can be found in reference [reference needed]. Figure 1 The relevant descriptions of steps 101, 102, 103, 104, 105, and 106 in the corresponding embodiments will not be repeated here.
[0086] In some embodiments of this disclosure, the tracing unit 206 is configured to: obtain the original response storage address of the model response; bind the content atom, the valid source field, the valid evidence field, the set of source nodes actually visible in the current model call, the graph version corresponding to the compilation unit, the compilation unit identifier, the running identifier, and the original response storage address together as tracing record information and store them in the content atom database.
[0087] In some embodiments of this disclosure, the large model generation content processing apparatus 200 further includes a review unit (not shown in the figure), which is configured to: in response to the short tag declaring the source being undecoded by the short tag mapping table, write the short tag declaring the source into the unparsed source field and prohibit writing the short tag declaring the source into the valid source field; in response to the short tag declaring the evidence identifier not being in the evidence allowed list, prohibit writing the evidence identifier declaring the evidence identifier into the valid evidence field, mark the content atom as requiring manual review, and prohibit the content atom from automatically transitioning to the verified state.
[0088] In some embodiments of this disclosure, the generation unit 202 is configured to: extract the source node identifier of each source node in the source node set, generate an initial short label based on a truncated hash or an auto-incrementing sequence; perform collision detection within the scope of the current compilation unit, and if a collision is detected, re-encode by extending the label characters or adding a disambiguation sequence number to ensure that the short label is unique within the scope of the current compilation unit, and generate a short label mapping table.
[0089] In some embodiments of this disclosure, the configuration unit 203 is configured to: configure a structured response mode when native structured output is supported, based on the interface capabilities of the target large language model; and configure a structured text template with start and end markers in the prompt words when native structured output is not supported. The receiving unit 204 is configured to: retain the original model response, perform format repair on the structural markers, fences, or punctuation of the response, and perform structural verification and source and evidence identification verification on the repaired response.
[0090] In some embodiments of this disclosure, the acquisition unit 201 is configured to: receive a compilation unit passed in by the pre-compiler; acquire a list of graph nodes actually written into the context by the pre-compiler in this model call, and use the list of graph nodes as the actual set of visible source nodes.
[0091] In some embodiments of this disclosure, the large model generation content processing device 200 further includes a recompilation unit (not shown in the figure). The recompilation unit is configured to: perform batch structure verification on multiple content atoms generated in the same model call of the current compilation unit and belonging to the same output slot; detect whether all the declaration source short tags of each content atom have corresponding entries in the short tag mapping table, and whether all the declaration evidence identifiers of each content atom exist in the evidence allowed list; in response to detecting that the declaration source short tag of any content atom cannot be decoded or the declaration evidence identifier is invalid, mark the entire set of content atoms corresponding to the compilation unit or output slot as a verification failure state, prohibit the set of content atoms from entering the downstream publishing process, and trigger manual review or automatic recompilation.
[0092] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0093] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0094] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their patterns are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0095] like Figure 3As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.
[0096] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0097] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the large model generation content processing method. For example, in some embodiments, the large model generation content processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the large model generation content processing method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform the large model generation content processing method by any other suitable means (e.g., by means of firmware).
[0098] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0099] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable large-scale model generation content processing apparatus, such that when executed by the processor or controller, the patterns / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0100] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0101] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0102] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0103] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0104] The foregoing description of specific exemplary embodiments of this disclosure is for illustrative and explanatory purposes. These descriptions are not intended to limit this disclosure to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of this disclosure and their practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of this disclosure, as well as various different choices and variations. The scope of this disclosure is intended to be defined by the claims and their equivalents.
Claims
1. A method for processing content generated from large models, the method comprising: Obtain the compilation unit to be executed and the set of source nodes that are actually visible to the compilation unit in this model call of the target large language model; Based on the source node set, a unique short tag mapping table and an evidence permission list are generated; the short tag mapping table is used to record the correspondence between source node identifiers and source short tags; the evidence permission list includes the evidence identifiers corresponding to the evidence nodes in the source node set that are allowed to be referenced by this compilation unit. According to the short label mapping table, the source node input representation of the current model call is appended with a corresponding source short label, and the input representation with the source short label and the evidence allowed list are configured in the request sent to the target large language model; and a structured output constraint including multiple output items is configured in the request so that the model response returned by the target large language model carries the declaration source short label and the declaration evidence identifier, wherein each output item includes a content field, an array of declaration source short labels and an array of declaration evidence identifiers; Receive the model response returned by the target large language model, parse each output item in the model response, and generate content atoms and atomic information of each content atom respectively; Obtain the short label of the claim source and the claim evidence identifier of each content atom in the plurality of content atoms; when the short label of the claim source is decoded through the short label mapping table, write the source node identifier corresponding to the short label of the claim source into the valid source field included in the content atom; when the claim evidence identifier belongs to the evidence allowed list, write the claim evidence identifier into the valid evidence field included in the content atom; After the content atoms that have completed the verification process and their verification status are bound to the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to this model call, they are persistently stored.
2. The method according to claim 1, characterized in that, The process of binding the completed verification atoms and their verification status with the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to this model call and then persistently storing them includes: Obtain the original response storage address of the model response; The content atom, the valid source field, the valid evidence field, the set of source nodes actually visible in this model call of the compilation unit corresponding to this model call, the graph version corresponding to the compilation unit, the compilation unit identifier, the running identifier, and the original response storage address are all bound together and stored in the content atom database as source tracing record information.
3. The method according to claim 1, characterized in that, The method further includes: In response to the fact that the declared source short tag cannot be decoded through the short tag mapping table, the declared source short tag is written to the unparsed source field, and writing the declared source short tag to the valid source field is prohibited; In response to the fact that the declared evidence identifier does not belong to the evidence allowed list, it is prohibited to write the declared evidence identifier into the valid evidence field, and the entire content atom is marked as requiring manual review, and the content atom is prohibited from automatically flowing to the verified status.
4. The method according to claim 1, characterized in that, The generation of a unique short label mapping table includes: Extract the source node identifier of each source node in the source node set, and generate an initial short label based on a truncated hash or an auto-incrementing sequence; Collision detection is performed within the scope of the current compilation unit. If a conflict is detected, the short tag is re-encoded by extending the tag character or adding a disambiguation number to ensure that the short tag is unique within the scope of the current compilation unit, and the short tag mapping table is generated.
5. The method according to claim 1, characterized in that, The structured output constraints configured in the request, which include multiple output items, include: Based on the interface capabilities of the target large language model, configure a structured response mode when native structured output is supported; When native structured output is not supported, configure a structured text template with start and end markers in the prompt words; The parsing of each output item in the model response includes: retaining the original model response, performing format repair on the structural markers, fences, or punctuation marks of the response, and performing structural verification and source and evidence identification verification on the repaired response.
6. The method according to claim 1, characterized in that, The process of obtaining the compilation unit to be executed and the set of source nodes actually visible to the compilation unit in this model call of the target large language model includes: Receives compilation units passed in by the pre-compiler; Obtain the list of graph nodes that the pre-compiler actually wrote into the context during this model call, and use the list of graph nodes as the actual set of visible source nodes.
7. The method according to any one of claims 1-6, characterized in that, The method further includes: Batch structure verification is performed on multiple content atoms generated in the same model call and belonging to the same output slot in this compilation unit. This checks whether all the short tags of the declaration source of each content atom have corresponding entries in the short tag mapping table, and whether all the declaration evidence identifiers of each content atom exist in the evidence allowed list. In response to the detection that the source tag of any content atom cannot be decoded or the claim evidence identifier is invalid, the entire set of content atoms corresponding to the compilation unit or the output slot is marked as failing the verification, the set of content atoms is prohibited from entering the downstream publishing process, and manual review or automatic recompilation is triggered.
8. A large model generation content processing apparatus, the apparatus comprising: The acquisition unit is configured to acquire the compilation unit to be executed and the set of source nodes that the compilation unit is actually visible in the current model call of the target large language model; The generation unit is configured to generate a unique short tag mapping table and an evidence permission list based on the source node set; the short tag mapping table is used to record the correspondence between source node identifiers and source short tags; the evidence permission list includes the evidence identifiers corresponding to the evidence nodes in the source node set that are allowed to be referenced by this compilation unit. The configuration unit is configured to attach corresponding source short tags to the source node input representation of the current model call according to the short tag mapping table, and configure the input representation with source short tags and the evidence allowed list into the request sent to the target large language model; and configure structured output constraints including multiple output items in the request so that the model response returned by the target large language model carries the declaration source short tags and declaration evidence identifiers, wherein each output item includes a content field, an array of declaration source short tags, and an array of declaration evidence identifiers; The receiving unit is configured to receive the model response returned by the target large language model, parse each output item in the model response, and generate content atoms and atomic information of each content atom respectively. The verification unit is configured to obtain the claim source short tag and claim evidence identifier of each content atom among the plurality of content atoms; when the claim source short tag is decoded through the short tag mapping table, the source node identifier corresponding to the claim source short tag is written into the valid source field included in the content atom; when the claim evidence identifier belongs to the evidence allowed list, the claim evidence identifier is written into the valid evidence field included in the content atom. The tracing unit is configured to persistently store the content atoms that have completed the verification process and their verification status, which are bound to the graph version, compilation unit identifier, and runtime identifier of the compilation unit corresponding to this model call.
9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
10. A computer-readable storage medium storing computer instructions thereon, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.
