A lightweight semantic labeling based llm streaming output multi-channel rendering method

Abstract

The present application relates to the technical field of natural language processing, and particularly relates to a LLM streaming output multi-channel rendering method based on light semantic marking, which injects semantic blocks in the LLM streaming output by defining a pair of start and end marking protocol carrying type identification, configures a character-level parsing state machine in the front end, detects the start marking feature trigger character, switches to the type parsing state to extract the identification, then enters the content receiving state to cache data, and ends the caching and switches back to the text state after detecting the end marking feature trigger character; for different semantic block types, the content is routed to independent rendering channels for parallel rendering; special code blocks are dispatched to pre-registered front-end components according to language labels for protocolized rendering. The present application effectively solves the problems of marking cross-packet truncation and semantic segmentation in streaming transmission, and realizes parallel streaming rendering and structured display of thinking, text and tool calling contents.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a multi-channel rendering method for LLM streaming output based on lightweight semantic tagging. Background Technology

[0002] With the widespread application of Large Language Models (LLM) in fields such as intelligent question answering, code generation, and automated operation and maintenance, streaming-based output methods (such as Server-Sent Events, SSE) have become the mainstream technology for improving user-perceived response speed. However, existing LLM streaming rendering solutions still have the following technical problems that urgently need to be solved:

[0003] First, the semantic stage is highly coupled with the rendering pipeline, resulting in a poor user experience. Existing technologies typically employ unified rendering after receiving the entire packet or simple splicing within a single text channel. This leads to the model's intermediate chain-of-thought process, tool call parameters, and the final response text being presented all mixed together. Users cannot distinguish between temporary reasoning processes and final results, and lengthy thought content occupies screen space, increasing the user's reading burden. Furthermore, it fails to achieve the folding and hiding of thought content and the real-time presentation of the main text.

[0004] Second, tag parsing lacks robustness and struggles to handle streaming packet-segmented scenarios. To achieve content differentiation, some solutions attempt to introduce tags for semantic segmentation. However, existing parsing logic is mostly based on regular expressions or fixed string matching, lacking adaptation to network transmission characteristics. In actual SSE transmission, a complete tag (such as...) <thinking>The data may be split into two packets and sent separately. Traditional line-level or block-level parsers are prone to parsing failures because they cannot recognize the truncation mark, which can lead to page rendering errors or content loss.

[0005] Third, the ability to display structured content is weak, and scalability is poor. For displaying complex structured data such as network topology and charts, existing technologies typically rely on hard-coded front-end judgments or require the back-end to return a specific JSON structure. This approach not only increases the coupling between the front-end and back-end but also fails to support the dynamic dispatch of code blocks in lightweight markup languages ​​such as Markdown. When users need to display ECharts charts or TCP topology, existing solutions lack a universal, protocol-based rendering mechanism, making it difficult to meet diverse display needs.

[0006] To address the aforementioned issues, Chinese patent CN121523718A (Qingchuang) discloses a type mapping scheme based on a configuration table, but it still relies on external configuration for type identification and does not solve the tag concatenation problem under streaming sub-packets. While Chinese patent CN121524453A (Yinlian) involves streaming processing, its tags are mainly used for citation indexing and cannot support complex semantic block type identifiers and nested structures. Therefore, there is an urgent need for a streaming output solution that supports low latency, multi-channel parallel rendering, and high fault tolerance. Summary of the Invention

[0007] The purpose of this invention is to provide a multi-channel rendering method for LLM streaming output based on lightweight semantic tagging, so as to solve the problems of difficult semantic segmentation, high rendering latency, and complex display of structured content in the streaming output scenario mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a multi-channel rendering method for LLM streaming output based on lightweight semantic markup, comprising:

[0009] Define a lightweight semantic tagging protocol, which includes a pair of configured start tags and end tags. The start tag carries a semantic block type identifier, and the end tag matches the start tag according to the semantic block boundary.

[0010] During the LLM streaming output process, different types of semantic blocks are injected into the output stream in an event-driven manner. The semantic blocks at least cover one of the following: thinking phase block, text phase block, and tool call phase block.

[0011] The front-end is configured with a character-level parsing state machine, denoted as commandParser. The state machine is in text state by default. When a feature trigger character of the start marker is detected in the text state, it switches to the type parsing state to extract the semantic block type identifier. Then it switches to the content receiving state to cache the current semantic block content. When a feature trigger character of the end marker is detected in the content receiving state, the caching of the current semantic block content ends and it switches back to the text state.

[0012] For different semantic block type identifiers, the corresponding semantic blocks are routed to independent rendering channels for parallel rendering;

[0013] For special code blocks in the output stream, the code block content is dispatched to a pre-registered front-end component for protocol-based rendering based on the code block's info-string language tag. The language tag includes at least one of tcplink and echarts.

[0014] Preferably, the start marker adopts the format !!!{type} and the end marker adopts the format ^^^{type}, where {type} is a semantic block type identifier, and the value includes at least one of think, text, and toolcall; the lightweight semantic tagging protocol also includes nested sub-agent tags, in the format SUBAGENT[[meta]]…end, used to identify the recursive call semantic block of the sub-agent.

[0015] Preferably, the state transition logic of the commandParser state machine further includes:

[0016] In type resolution mode, if no valid {type} semantic block type identifier is extracted, the detected start marker feature trigger character will be output as ordinary text, and the process will fall back to text mode.

[0017] In the content receiving state, if a new start marker feature trigger character is detected, the caching of the current semantic block is forcibly terminated, the cached content is submitted to the corresponding rendering channel, and then the type resolution process of the new semantic block is started.

[0018] Preferably, the rendering strategy of the independent rendering channel is bound to the semantic block type identifier:

[0019] When the semantic block type is think, the corresponding rendering channel is displayed in a collapsed state by default, and is only expanded when the user triggers an interaction;

[0020] When the semantic block type is text, the corresponding rendering channel adopts a word-by-word / segment-by-segment streaming rendering strategy to maintain the continuity of content output;

[0021] When the semantic block type is toolcall, the corresponding rendering channel is displayed in expanded state by default, synchronizing the rendering tool call parameters and execution state;

[0022] When the semantic block is a nested semantic block marked with SUBAGENT[[meta]]...end, the corresponding rendering channel adopts an indented hierarchical display, and the hierarchy depth is positively correlated with the sub-agent call depth.

[0023] Preferably, the protocol-based rendering logic of the special code block includes:

[0024] A front-end pre-built mapping table of language tags and rendering components is provided. In the mapping table, the tcplink tag corresponds to the TCP network topology component, the echart tag corresponds to the ECharts visualization component, and additional support is provided for the mapping of table components.

[0025] When the info-string of a special code block matches any language tag in the mapping table, the corresponding component is invoked to parse and render the code block content. After rendering is complete, the component is mounted under the currently active rendering channel.

[0026] Preferably, the meta field in the SUBAGENT[[meta]]…end tag includes at least one of the following: a unique identifier of the sub-agent, a call depth, an execution status code, and an input / output digest, to support the tracing of the semantic blocks and the tracking of their states.

[0027] Preferably, the transmission protocol of the output stream is Server-SentEvents (SSE) or WebSocket, and the commandParser state machine supports character concatenation triggered by the tag feature across data packets. When the end of a single data packet contains only a partial prefix of the start tag or end tag, the prefix is ​​cached until the next data packet arrives and the tag integrity verification is completed.

[0028] A lightweight semantic markup-based LLM streaming multi-channel rendering apparatus includes:

[0029] The tagging protocol definition module is used to define a lightweight semantic tagging protocol that includes a pair of start tags and end tags, wherein the start tag carries a semantic block type identifier;

[0030] The streaming injection module is used to inject different types of semantic blocks into the output stream in an event-driven manner during the LLM streaming output process.

[0031] The state machine parsing module, denoted as commandParser, is configured as a character-level parsing state machine. It is used to detect the feature trigger character of the start marker in the text state and switch to the type parsing state. After extracting the semantic block type identifier, it switches to the content receiving state. After detecting the feature trigger character of the end marker in the content receiving state, it ends the current semantic block and switches back to the text state.

[0032] The multi-channel rendering module is used to route the corresponding semantic blocks to independent rendering channels for parallel rendering based on different semantic block type identifiers.

[0033] The code block dispatch module is used to dispatch the content of special code blocks in the output stream to pre-registered front-end components for protocol-based rendering based on the code block's info-string language tag.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] 1. This invention achieves integrated single-model calculation for watershed simulation under the influence of dams and gates at the architectural level, without coupling independent hydrodynamic models, water quality models, or external scheduling modules. By pre-setting the dam and gate locations as sub-watershed outlets and reconstructing spatial discretization, a spatial mapping relationship of "dam and gate control section as sub-watershed control outlet" is established. River confluence, water level calculation, dam and gate outflow, water balance, and water quality degradation are all integrated into the same model and completed within the same calculation time step. This fundamentally solves the problems of complex structure, difficult interface development, mismatch of spatiotemporal scales, and data exchange lag in existing multi-model coupling methods. It significantly reduces the workload of model construction, parameter calibration, and operation and maintenance, and realizes dynamic synchronous updates of water quantity and water quality processes, which can truly reflect the real-time control effect of dam and gate opening and closing and changes in opening degree.

[0036] 2. This invention improves the physical rationality and reliability of simulation results from the perspective of computational mechanisms. Based on the energy-momentum equation, it directly calculates the outflow process of dams under different hydraulic conditions, and automatically distinguishes between free outflow and submerged outflow states according to the upstream and downstream water levels. It fully preserves the physical mechanism of dam discharge and avoids the problem of missing physical mechanisms caused by empirical formulas, machine learning mapping, or virtual reservoir generalization. At the same time, all calculation variables are internal variables of the model, eliminating the need for cross-model data transfer, fundamentally eliminating the problem of multi-model error accumulation along the process. With the unified parameter calibration system, it effectively reduces the uncertainty of simulation results, makes the physical meaning of parameters clearer, and significantly improves simulation accuracy and reliability.

[0037] 3. This invention improves simulation efficiency and engineering practicality at the application level. It adopts an integrated calculation process based on river network topology, eliminating the need for multiple model iterations and significantly improving computational efficiency. This can meet the rapid simulation needs of the impact of multiple dams on long series and large watershed scales. Actual watershed applications show that this method can accurately reflect the regulation and peak-shaving effect of dams on pollution loads. The simulated total nitrogen and total phosphorus output processes are more consistent with actual laws. It can quantitatively evaluate the water volume and pollution load response under different dam scheduling schemes, providing reliable scientific support for the formulation of dam ecological scheduling schemes and the management of watershed pollution loads. Attached Figure Description

[0038] Figure 1 This is a system architecture diagram of an LLM streaming multi-channel rendering method based on lightweight semantic tagging, according to the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] This embodiment is applied to a web-based AI intelligent agent dialogue system. The system is built on a Browser / Server architecture. The front end uses the React framework to render the user interface, and the back end uses the Python FastAPI framework to build an LLM (Large Language Model) inference service. The front end and the back end establish a Server-Sent Events (SSE) long connection through the HTTP protocol to realize the streaming push of LLM-generated content.

[0041] Example 1: Definition of Lightweight Semantic Tag Protocol

[0042] To address the parsing latency issues caused by the cumbersome tagging protocols and reliance on external configuration mapping tables for semantic types in existing technologies, this application defines a lightweight semantic tagging protocol with self-portable types. This protocol directly encodes the semantic block type through the tag string itself, eliminating the need for front-end preloading of JSON configuration files for type lookup, thereby significantly reducing parsing latency.

[0043] The protocol includes two types of core tag structures:

[0044] Stage semantic block tagging pairs: Use !!!{type} as the start tag and ^^^{type} as the end tag. Wherein, {type} is the semantic block type identifier, used to indicate the stage attribute of the semantic block in the AI ​​inference process. This embodiment supports, but is not limited to, the following values:

[0045] think: corresponds to the thought chain (CoT) output of the AI ​​model, which belongs to the intermediate reasoning process;

[0046] text: The final response text presented to the user by the AI ​​model;

[0047] toolcall: The parameters, execution status, and return results of the AI ​​model calling external tools.

[0048] Example: !!!The think user needs to query IP connectivity, I will perform a Ping operation^^^think defines a complete semantic block for the thinking phase.

[0049] Nested sub-agent markers: Use the SUBAGENT[[meta]]…end format. This marker supports recursive call semantics in multi-agent collaborative scenarios. [[meta]] is a metadata encapsulation field used to store the dynamic execution context of the sub-agent. This is fundamentally different from the [[number]] markers used in existing technologies to identify static citations: the meta field in this scheme stores dynamic, structured runtime data (such as call ID, depth, and state), while the citation marker only stores static document index values.

[0050] Example: SUBAGENT[[{"id":"sub_001","depth":1,"status":"running"}]] is probing the gateway...end.

[0051] To ensure protocol compatibility, this embodiment supports custom configuration of the marker prefix. For example, users can change the start marker prefix from !!! to @@@ and the end marker prefix from ^^^ to &&& through front-end configuration, requiring only a synchronous update of the feature-triggered character matching logic in the front-end parser.

[0052] Example 2: Semantic Block Streaming Injection Mechanism

[0053] In the LLM streaming output process, to achieve precise segmentation of semantic blocks, this embodiment constrains the LLM output format through the Prompt project. The system prompt forcibly embeds format constraint instructions, guiding the LLM to generate content according to the protocol defined in Embodiment 1.

[0054] After receiving the token generated by the LLM, the backend service does not perform complete concatenation; instead, it directly encapsulates it into an SSE event stream and pushes it to the frontend in real time. The data format of each SSE event is as follows:

[0055] event: token

[0056] data: {"chunk": "!!!think first analyzes the user intent: query network status^^^think"}

[0057] The frontend listens to this streaming channel through the EventSource interface and sends the received chunk data fragments to the commandParser state machine for processing in real time. This event-driven injection method ensures low-latency data transmission from the source to the parser.

[0058] Example 3: Specific Implementation of the commandParser State Machine

[0059] To address the rendering issues caused by packet truncation markers and the aliasing of different semantic blocks in streaming transmission, this embodiment constructs a character-level parsing state machine at the front end, denoted as commandParser. This state machine maintains its internal state to achieve real-time and accurate segmentation of the streaming character stream.

[0060] 3.1 State Definition and Variable Initialization

[0061] commandParser defines three core runtime states to handle different parsing scenarios:

[0062] TEXT (Text Mode): The default mode, used to process ordinary text content or non-marked characters.

[0063] PARSING_TYPE (Type resolution state): Start tag prefix detected, resolving the {type} field to determine the semantic block type.

[0064] BUFFERING_CONTENT (Content Cache State): A valid semantic block type has been identified, and the specific content of the semantic block is being cached, awaiting the end marker.

[0065] The following example uses JavaScript to demonstrate the initialization logic of the core data structure of a state machine:

[0066] class CommandParser {

[0067] constructor() {

[0068] this.states = {

[0069] TEXT: 'text',

[0070] PARSING_TYPE: 'parsing_type',

[0071] BUFFERING_CONTENT: 'buffering_content'

[0072] };

[0073] this.currentState = this.states.TEXT;

[0074] this.typeBuffer = ''; / / Type resolution buffer: temporarily stores the {type} character to be resolved.

[0075] this.contentBuffer = ''; / / Content buffer: temporarily stores the content of the current semantic block.

[0076] this.currentType = null; / / The type of the currently resolved semantic block

[0077] this.pendingMarkerPrefix = ''; / / Prefix buffer to be matched: used for splicing markers across packets.

[0078] / / Multi-channel rendering manager reference

[0079] this.renderChannels = getRenderChannels();

[0080] }

[0081] / / ... Follow-up processing methods

[0082] }

[0083] Those skilled in the art will know that the above logic can also be implemented using TypeScript, Python, or Go, and all of these fall within the scope of protection of this invention.

[0084] 3.2 Character-by-character parsing logic

[0085] The state machine scans each input character and executes the corresponding action based on the current state. The following logic solves the technical problems of cross-packet concatenation marking and exception marking fault tolerance:

[0086] handleChar(char) {

[0087] switch (this.currentState) {

[0088] case this.states.TEXT:

[0089] / / Detecting the start marker feature trigger character '!' in text mode

[0090] if (char === '!') {

[0091] this.pendingMarkerPrefix += char;

[0092] / / If the prefix accumulates to '!!!', then switch to type resolution mode.

[0093] if (this.pendingMarkerPrefix === '!!!') {

[0094] this.currentState = this.states.PARSING_TYPE;

[0095] this.typeBuffer = '';

[0096] }

[0097] } else {

[0098] / / For characters other than '!', output plain text and clear the prefix to be matched.

[0099] this.flushPendingPrefix();

[0100] this.renderChannels.text.appendChar(char);

[0101] }

[0102] break

[0103] case this.states.PARSING_TYPE:

[0104] / / Accumulate alphanumeric characters to resolve type

[0105] if ( / [a-zA-Z] / .test(char)) {

[0106] this.typeBuffer += char;

[0107] } else {

[0108] const validTypes = ['think', 'text', 'toolcall'];

[0109] / / Check if the parsed type is valid

[0110] if (validTypes.includes(this.typeBuffer)) {

[0111] this.currentType = this.typeBuffer;

[0112] this.currentState = this.states.BUFFERING_CONTENT;

[0113] this.contentBuffer = '';

[0114] } else {

[0115] / / Illegal type, fallback: Output the detected '!!!' + typeBuffer as plain text.

[0116] this.flushToText(`!!!${this.typeBuffer}`);

[0117] this.currentState = this.states.TEXT;

[0118] }

[0119] }

[0120] break

[0121] case this.states.BUFFERING_CONTENT:

[0122] / / Detecting the end marker feature triggers the character '^'

[0123] if (char === '^') {

[0124] this.pendingMarkerPrefix += char;

[0125] if (this.pendingMarkerPrefix === '^^^') {

[0126] / / Upon matching the end marker, submit the content to the corresponding rendering channel.

[0127] this.renderChannels[this.currentType].commit(this.contentBuffer);

[0128] / / Reset the state machine

[0129] this.currentState = this.states.TEXT;

[0130] this.currentType = null;

[0131] this.contentBuffer = '';

[0132] this.pendingMarkerPrefix = '';

[0133] }

[0134] } else {

[0135] / / Handling cross-packet truncation: If '^' was previously cached but is not actually the end marker, then re-paste it.

[0136] if (this.pendingMarkerPrefix.length > 0) {

[0137] this.contentBuffer += this.pendingMarkerPrefix;

[0138] this.pendingMarkerPrefix = '';

[0139] }

[0140] this.contentBuffer += char;

[0141] / / A new start marker is detected in content mode, forcibly ending the current block (supports marker nesting / interruption)

[0142] if (char === '!') {

[0143] this.pendingMarkerPrefix = '!';

[0144] }

[0145] }

[0146] break

[0147] }

[0148] }

[0149] Based on the above logic, when SSE transmission causes !!!think to be split into two data packets (e.g., the first packet ends with !! and the second packet begins with !think), the pendingMarkerPrefix buffer can cache the incomplete marker prefix and complete the splicing after the next data packet arrives, thereby realizing the integrity verification of cross-packet markers.

[0150] Example 4: Multi-channel rendering strategy

[0151] This embodiment establishes independent rendering pipelines for different semantic block types to address the UI congestion and poor user experience caused by a single rendering pipeline. The rendering strategy for each pipeline is tied to the technical characteristics of the semantic block.

[0152] Think Channel: Since the thinking process is intermediate and not the user's focus, this channel is displayed in a collapsed state by default, showing only placeholders. This avoids lengthy thinking processes taking up screen space and reduces the rendering pressure on the first screen. Users can expand to view details by clicking the placeholder.

[0153] Text channel: This channel carries the core response content and uses a character-by-character streaming rendering strategy. Characters are immediately appended to the DOM node after being submitted from the buffer, simulating the effect of manual typing and significantly reducing user-perceived latency (first character delay).

[0154] Toolcall channel: This channel features structured content and defaults to an expanded display. The rendering logic synchronously processes the tool name, JSON parameters, and execution status (loading animation / success icon / error code), enhancing the transparency of toolcalls.

[0155] Subagent Channel: This channel uses an indented hierarchical display. The indentation is determined by the depth field in the meta tag (the indentation increases by 20px for every 1 increment in depth), intuitively reflecting the hierarchical relationship of the Multi-Agent calls.

[0156] Example 5: Protocol-based rendering of special code blocks

[0157] To support the dynamic rendering of complex structured content such as TCP topology and ECharts, this embodiment implements a protocol-based dispatch mechanism based on code block language tags. This mechanism improves the scalability of the front-end architecture by decoupling content parsing from component rendering.

[0158] Mapping table between pre-built language tags and rendering components in the front end:

[0159] const RENDERER_MAP = {

[0160] 'tcplink': TCPTopologyRenderer, / / TCP network topology component

[0161] 'echart': EChartsRenderer, / / ECharts visualization component

[0162] 'table': TableRenderer, / / Table component

[0163] 'mermaid': MermaidRenderer / / Mermaid flowchart component

[0164] };

[0165] The parsing process is as follows:

[0166] When the commandParser is in text mode and detects the Markdown code block start character ```, it extracts the immediately following info-string as a language tag (such as tcplink).

[0167] Query RENDERER_MAP; if the language tag exists, instantiate the corresponding rendering component.

[0168] The text content (such as JSON configuration) within the code block is passed to the component for parsing.

[0169] After the component renders and generates a virtual DOM node, it is mounted under the currently active rendering channel.

[0170] Technical benefits: Compared to existing technologies that require hard-coding of content types on the front end, this protocol-based dispatch mechanism allows adding a new type of structured content (such as a mind map) to simply register a new mapping relationship without modifying the core parsing logic, thus reducing system coupling.

[0171] 5.1 Example of Rendering a TCP Link Code Block

[0172] For the following code block:

[0173] {"nodes":[{"id":"pc"},{"id":"server"}],"edges":[{"source":"pc","target":"server"}]}

[0174] After parsing the JSON data, TCPTopologyRenderer generates an SVG or Canvas canvas containing nodes and connections, allowing users to drag, zoom, and interact with the data to intuitively display network connections.

[0175] 5.2 Example of ECharts code block rendering

[0176] For the following code block:

[0177] {"title":{"text":"Delay Statistics"},"xAxis":{"data":["1","2","3"]},"series":[{"type":"line","data":[12,15,13]}]}

[0178] EChartsRenderer calls the ECharts library to instantiate chart objects, visualizing data as line charts and supporting mouse hover to view specific values.

[0179] Example 6: End-to-End Process Example

[0180] The following example, using a complete Ping detection dialogue, demonstrates the collaborative process of SSE stream, state machine transitions, and rendering effects:

[0181] SSE stream output:

[0182] event: token

[0183] data: {"chunk": "!!!think user needs to check IP connectivity and decides to call the ping tool^^^think"}

[0184] event: token

[0185] data: {"chunk": "!!!text detection complete, results are as follows:\n```tcplink\n{\"nodes\":[{\"id\":\"pc\"},{\"id\":\"server\"}]}\n```\n^^^text"}

[0186] event: token

[0187] data: {"chunk": "SUBAGENT[[{\"id\":\"sub_001\",\"depth\":1,\"status\":\"success\"}]]Ping successful, packet loss rate 0% end"}

[0188] State machine transitions:

[0189] Receive !!!think, switch from TEXT state to PARSING_TYPE state, parse type=think, switch to BUFFERING_CONTENT state to cache content.

[0190] Receive ^^^think, submit content to the thinking channel, and switch back to TEXT mode.

[0191] Receive !!!text, parse the type, and cache the main content.

[0192] Upon parsing the 'tcplink', protocol-based dispatch is triggered, and TCPTopologyRenderer is instantiated.

[0193] Receive ^^^text and submit the main text to the main text channel.

[0194] Receive SUBAGENT[[...]], parse the metadata, and submit the content to the sub-agent channel.

[0195] Final rendering result:

[0196] The page first presents a collapsed entry point for the thought process; then the main text displays "Detection complete..." word by word; below the main text, a network topology diagram is rendered in real time; below the topology diagram, indented sub-agent execution results are displayed. Compared to traditional full-package rendering solutions, this solution reduces the interactive time on the first screen by approximately 75%.

[0197] Example 7: Abnormal Scene Handling

[0198] This embodiment also considers abnormal scenarios in real-world network environments, improving the robustness of the solution:

[0199] Trace spelling error: If the LLM outputs !!!tink...^^^tink (think is misspelled), the state machine fails to validate in the PARSING_TYPE state. !!!tink will be output as plain text, which will not affect the parsing of subsequent correct tags.

[0200] JSON parsing failure: If the JSON format within the tcplink code block is incorrect, the rendering component will capture the exception and display a "Topology rendering failed" placeholder, which will not affect the normal rendering of other channels.

[0201] Connection interruption: If the SSE connection is interrupted during the tag transmission, the state machine will automatically clear the buffer after the timeout and display "Connection interrupted" on the UI to avoid the page freezing.

[0202] Extended Implementation

[0203] Those skilled in the art will understand that the above embodiments are merely preferred embodiments of the present invention. Without departing from the core concept of the present invention, the following extensions can be made:

[0204] Tag Protocol Extension: Added the {type} value, such as image to wrap Base64 encoded image data, and audio to wrap audio streams. Only the corresponding rendering channel needs to be added.

[0205] Transport protocol extension: Replacing SSE with WebSocket protocol only requires adjusting the data input source of commandParser to the WebSocket message listener; the core logic of the state machine remains unchanged.

[0206] Multi-platform adaptation: To port the above front-end logic to mobile devices (React Native / Flutter), you only need to replace DOM operations with native view operations, and the parsing and rendering logic can be completely reused.

[0207] The present invention effectively solves the technical problems of difficult semantic segmentation, high rendering latency, and complex display of structured content in streaming output scenarios through the above technical solution, and realizes low latency, multi-channel, and protocol-based presentation of LLM output content.

[0208] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.< / thinking>

Claims

1. A multi-channel rendering method for LLM streaming output based on lightweight semantic markup, characterized in that, include: Define a lightweight semantic tagging protocol, which includes a pair of configured start tags and end tags. The start tag carries a semantic block type identifier, and the end tag matches the start tag according to the semantic block boundary. During the LLM streaming output process, different types of semantic blocks are injected into the output stream in an event-driven manner. The semantic blocks at least cover one of the following: thinking phase block, text phase block, and tool call phase block. The front-end is configured with a character-level parsing state machine, denoted as commandParser. The state machine is in text state by default. When a feature trigger character of the start marker is detected in the text state, it switches to the type parsing state to extract the semantic block type identifier. Then it switches to the content receiving state to cache the current semantic block content. When a feature trigger character of the end marker is detected in the content receiving state, the caching of the current semantic block content ends and it switches back to the text state. For different semantic block type identifiers, the corresponding semantic blocks are routed to independent rendering channels for parallel rendering; For special code blocks in the output stream, the code block content is dispatched to a pre-registered front-end component for protocol-based rendering based on the code block's info-string language tag. The language tag includes at least one of tcplink and echarts.

2. The method for multi-channel LLM streaming output based on lightweight semantic markup as described in claim 1, characterized in that, The start marker adopts the format !!!{type} and the end marker adopts the format ^^^{type}, where {type} is a semantic block type identifier, and the value includes at least one of think, text, and toolcall; the lightweight semantic tagging protocol also includes nested sub-agent tags, in the format SUBAGENT[[meta]]…end, which are used to identify the recursive call semantic blocks of sub-agents.

3. The method for multi-channel LLM streaming output based on lightweight semantic markup as described in claim 2, characterized in that, The state transition logic of the commandParser state machine also includes: In type resolution mode, if no valid {type} semantic block type identifier is extracted, the detected start marker feature trigger character will be output as ordinary text, and the process will fall back to text mode. In the content receiving state, if a new start marker feature trigger character is detected, the caching of the current semantic block is forcibly terminated, the cached content is submitted to the corresponding rendering channel, and then the type resolution process of the new semantic block is started.

4. The method for multi-channel LLM streaming output based on lightweight semantic markup as described in claim 1, characterized in that, The rendering strategy of the independent rendering pipeline is bound to the semantic block type identifier: When the semantic block type is think, the corresponding rendering channel is displayed in a collapsed state by default, and is only expanded when the user triggers an interaction; When the semantic block type is text, the corresponding rendering channel adopts a word-by-word / segment-by-segment streaming rendering strategy to maintain the continuity of content output; When the semantic block type is toolcall, the corresponding rendering channel is displayed in expanded state by default, synchronizing the rendering tool call parameters and execution state; When the semantic block is a nested semantic block marked with SUBAGENT[[meta]]...end, the corresponding rendering channel adopts an indented hierarchical display, and the hierarchy depth is positively correlated with the sub-agent call depth.

5. The method for multi-channel LLM streaming output based on lightweight semantic markup as described in claim 1, characterized in that, The protocol-based rendering logic of the special code block includes: A front-end pre-built mapping table of language tags and rendering components is provided. In the mapping table, the tcplink tag corresponds to the TCP network topology component, the echart tag corresponds to the ECharts visualization component, and additional support is provided for the mapping of table components. When the info-string of a special code block matches any language tag in the mapping table, the corresponding component is invoked to parse and render the code block content. After rendering is complete, the component is mounted under the currently active rendering channel.

6. The method for multi-channel LLM streaming output based on lightweight semantic markup according to claim 2, characterized in that, The meta field in the SUBAGENT[[meta]]…end tag contains at least one of the following: a unique identifier for the sub-agent, a call depth, an execution status code, and an input / output digest, which is used to support the tracing of the source and state of the sub-agent's semantic blocks.

7. The method for multi-channel LLM streaming output based on lightweight semantic markup according to claim 1, characterized in that, The output stream is transmitted using Server-SentEvents (SSE) or WebSocket. The commandParser state machine supports character concatenation triggered by the tag feature across data packets. When a single data packet ends with only a partial prefix of the start tag or end tag, the prefix is ​​cached until the next data packet arrives and the tag integrity check is completed.

8. A lightweight semantic markup-based LLM streaming output multi-channel rendering apparatus as described in any one of claims 1-7, characterized in that, include: The tagging protocol definition module is used to define a lightweight semantic tagging protocol that includes a pair of start tags and end tags, wherein the start tag carries a semantic block type identifier; The streaming injection module is used to inject different types of semantic blocks into the output stream in an event-driven manner during the LLM streaming output process. The state machine parsing module, denoted as commandParser, is configured as a character-level parsing state machine. It is used to detect the feature trigger character of the start marker in the text state and switch to the type parsing state. After extracting the semantic block type identifier, it switches to the content receiving state. After detecting the feature trigger character of the end marker in the content receiving state, it ends the current semantic block and switches back to the text state. The multi-channel rendering module is used to route the corresponding semantic blocks to independent rendering channels for parallel rendering based on different semantic block type identifiers. The code block dispatch module is used to dispatch the content of special code blocks in the output stream to pre-registered front-end components for protocol-based rendering based on the code block's info-string language tag.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the LLM streaming output multichannel rendering method based on lightweight semantic tagging as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the LLM streaming multichannel rendering method based on lightweight semantic tagging as described in any one of claims 1 to 7.

Images