A streaming data processing method and system based on an intermediate layer

Abstract

This specification provides a method and system for streaming data processing based on an intermediate layer. The method is applied to an intermediate layer deployed between a server and a third-party large model. The method includes: receiving streaming text data from the third-party large model; identifying resource reference tags based on the streaming text data to determine resource addresses corresponding to the resource reference tags; generating placeholder identifiers corresponding to the resource addresses and replacing the resource reference tags with placeholder tags, the placeholder tags including the placeholder identifiers; forwarding the placeholder tags and the streaming text data to a client; wherein the client is configured to display the streaming text data in real time on a display page, and to locate and update placeholder elements corresponding to image positions on the display page based on the placeholder tags.

Description

Technical Field

[0001] This specification relates to the field of data processing technology, and in particular to a streaming data processing method and system based on an intermediate layer. Background Technology

[0002] With the rapid development of artificial intelligence technology, the construction industry is gradually introducing intelligent question-answering systems to assist on-site construction personnel in quickly resolving various technical issues. Currently, the industry generally adopts a hybrid architecture model of "third-party large-scale model application programming interface (API) and locally built building code knowledge base". Specifically, construction personnel can input construction technical questions through mobile apps or web platforms, such as "How to set up the stirrup reinforcement zone of frame beams?". The backend system (i.e., the server) first relies on the enterprise's internally built building code knowledge base (the data sources mainly include authoritative technical documents such as "16G101 Atlas" and "Code for Design of Concrete Structures", which are structured to form a searchable dataset) to perform retrieval-augmented generation (RAG) operations. This process aims to accurately retrieve text fragments of code clauses highly relevant to the question from massive unstructured documents, and simultaneously extract corresponding image resources such as detailed construction drawings, node detail drawings, and parameter tables. Subsequently, the system uses the retrieved text fragments as contextual information, along with the user's question, and submits them to a third-party Artificial Intelligence (AI) large-scale model API (such as OpenAI GPT-4, Anthropic Claude, etc.) for processing. Based on their powerful natural language understanding and generation capabilities, these third-party models transform obscure and difficult-to-understand technical specifications into easily understandable natural language solutions, effectively lowering the barrier for frontline construction personnel to access and understand technical standards.

[0003] To further enhance the user experience of human-computer interaction, current mainstream technical solutions widely adopt the SSE (Server-Sent Events) streaming protocol. Under this mechanism, when generating the final answer, third-party AI models no longer follow the traditional "all-in-one return after complete generation" model, but instead adopt a streaming output, i.e., a generation-as-you-go approach. Each time the third-party AI model generates a small piece of text (usually a word, a phrase, or a sentence), it immediately pushes the data block to the server via the SSE channel. The server then forwards these continuous text fragments to the front-end application. After receiving the SSE data stream, the front-end application uses a Markdown parser to render the received content in real time and display it on the user interface. This approach allows users to "read while waiting," significantly alleviating the anxiety caused by network latency or model computation time, and achieving a smoother human-computer dialogue experience.

[0004] In the implementation details of existing technologies, the standardized data retrieved by the RAG retrieval mode is typically temporarily stored and transmitted in a structured format of "text description and image URL". For example, a typical search result might be presented as: "The length of the stirrup reinforcement zone at the beam end is twice the beam height". See the image gallery description for details. To enable third-party AI models to appropriately reference and display these key image resources in their generated natural language responses, the system embeds specific formatting instructions in the system prompt when calling the API. These instructions require the third-party model to include the relevant context information. When the tag is executed, the content is converted into a Markdown-standard image URL for output. After parsing and understanding this system-level instruction, the third-party large model will automatically generate Markdown-compliant image reference tags in its SSE streaming output text, for example: "...the length of the encrypted area should be twice the beam height, and its specific structure is shown in the figure http: / / cdn.example.com / 16g101_fig3_2.png...". The front-end page's Markdown rendering engine (such as marked.js, markdown-it, and other open-source libraries) will automatically compile these tags into HTML after parsing them. Tags. The front-end application then loads and renders the corresponding specification diagrams based on the URL within the tag, allowing users to intuitively view the accompanying technical drawings while reading the text explanations, greatly enhancing the completeness and effectiveness of technical information delivery.

[0005] Therefore, it is necessary to provide a streaming data processing method and system based on an intermediate layer to transmit the deep association information between images and standard document chapters in the local knowledge base to the front-end user in real time, thereby solving the pain point problem of the separation between images and standard documents in the existing technology. Summary of the Invention

[0006] This specification provides one or more embodiments of a streaming data processing method based on an intermediate layer. The method is applied to an intermediate layer deployed between a server and a third-party large model. The method includes: receiving streaming text data from the third-party large model; identifying resource reference tags based on the streaming text data to determine resource addresses corresponding to the resource reference tags; generating placeholder identifiers corresponding to the resource addresses and replacing the resource reference tags with placeholder tags, the placeholder tags including the placeholder identifiers; forwarding the placeholder tags and the streaming text data to a client; wherein the client is configured to display the streaming text data in real time on a display page, and to locate and update placeholder elements corresponding to image positions on the display page based on the placeholder tags.

[0007] This specification provides one or more embodiments of a streaming data processing system based on an intermediate layer. The system includes an intermediate layer deployed between a server and a third-party large model. The intermediate layer includes: a receiving module configured to receive streaming text data from the third-party large model; an address determination module configured to identify resource reference tags based on the streaming text data and determine resource addresses corresponding to the resource reference tags; an identifier generation module configured to generate placeholder identifiers corresponding to the resource addresses and replace the resource reference tags with placeholder tags, the placeholder tags including the placeholder identifiers; and a forwarding module configured to forward the placeholder tags and the streaming text data to a client. The client is configured to display the streaming text data in real time on a display page and to locate and update placeholder elements corresponding to image positions on the display page based on the placeholder tags.

[0008] This specification provides one or more embodiments of an intermediate layer device for streaming data processing. The intermediate layer device is deployed between a server and a third-party large model. The intermediate layer device includes: a receiving layer configured to receive streaming text data from the third-party large model; a parsing layer configured to identify resource reference tags based on the streaming text data, determine resource addresses corresponding to the resource reference tags, generate placeholder identifiers corresponding to the resource addresses, and replace the resource reference tags with placeholder tags, the placeholder tags including the placeholder identifiers; and a forwarding layer configured to forward the placeholder tags and the streaming text data to a client. The client is configured to display the streaming text data in real time on a display page, and to locate and update placeholder elements corresponding to image positions on the display page based on the placeholder tags. Attached Figure Description

[0009] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 This is an exemplary block diagram of a streaming data processing system based on an intermediate layer, as shown in some embodiments of this specification. Figure 2 This is an exemplary flowchart of a streaming data processing method based on an intermediate layer, as shown in some embodiments of this specification. Detailed Implementation

[0010] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0011] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0012] Unless the context clearly indicates an exception, words such as "a," "an," "a kind," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0013] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0014] Figure 1 This is an exemplary block diagram of a streaming data processing system based on an intermediate layer, as shown in some embodiments of this specification.

[0015] In some embodiments, such as Figure 1 As shown, a streaming data processing system 100 based on an intermediate layer (hereinafter referred to as the system) may include an intermediate layer deployed between a server and a third-party large model. The intermediate layer includes a receiving module 110, an address determination module 120, an identifier generation module 130, and a forwarding module 140.

[0016] A middleware tier is an independent functional component deployed between a server and a third-party large model, used to process and forward data between the two. For example, a middleware tier could be a standalone microservice or an application deployed on a dedicated server, whose role is to intercept and process the data stream returned by the large model, and then forward the processed data to the client.

[0017] Third-party large models refer to large-scale artificial intelligence models developed and provided by entities independent of System 100. For example, third-party large models can be GPT series models (such as GPT-4), Gemini models, Claude models, DeepSeek models, etc.

[0018] In some embodiments, the receiving module 110 is configured to receive streaming text data from a third-party large model.

[0019] In some embodiments, the address determination module 120 is configured to identify resource reference tags based on streaming text data and determine the resource address corresponding to the resource reference tag.

[0020] In some embodiments, the identifier generation module 130 is configured to generate a placeholder identifier corresponding to a resource address and replace the resource reference label with a placeholder label, wherein the placeholder label includes the placeholder identifier.

[0021] In some embodiments, the forwarding module 140 is configured to forward placeholder tags and streaming text data to a client; wherein the client is configured to display streaming text data in real time on a display page, and to locate and update placeholder elements corresponding to image positions on the display page based on the placeholder tags.

[0022] For further explanation of the above content, please see [link / reference]. Figure 2 And related content.

[0023] It should be understood that Figure 1 The system and its modules shown can be implemented in various ways. For example, in some embodiments, Figure 1 The system and its modules shown can be implemented by a processor. The processor can process data and / or information obtained from other devices or system components. Based on this data, information, and / or processing results, the processor can execute program instructions to perform one or more functions described in this specification. In some embodiments, the processor may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-chip processing device). By way of example only, the processor may include a central processing unit (CPU), a controller, a microprocessor, or any combination thereof. In some embodiments, the processor may include multiple modules, each of which can be used to execute different program instructions.

[0024] It should be noted that the above description of the system and its modules is for convenience only and should not be construed as limiting this specification to the scope of the illustrated embodiments. It is understood that those skilled in the art, after understanding the principles of the system, may arbitrarily combine the various modules or construct subsystems connected to other modules without departing from these principles. In some embodiments, Figure 1 The receiving module, address determination module, identifier generation module, and forwarding module disclosed herein can be different modules within a single system, or a single module can implement the functions of two or more of the aforementioned modules. For example, the modules can share a single storage module, or each module can have its own separate storage module. Such variations are all within the scope of protection of this specification.

[0025] Figure 2 This is a flowchart illustrating a streaming data processing method based on an intermediate layer, according to some embodiments of this specification. For example... Figure 2 As shown, process 200 includes the following steps. In some embodiments, process 200 may be executed by an intermediate layer deployed between a server and a third-party large model in a middle-layer-based streaming data processing system 100.

[0026] In some embodiments, the method is applied to an intermediate layer deployed between a server and a third-party large model. The method includes: receiving streaming text data from the third-party large model; identifying resource reference tags based on the streaming text data to determine the resource address corresponding to the resource reference tag; generating a placeholder identifier corresponding to the resource address and replacing the resource reference tag with the placeholder tag, the placeholder tag including the placeholder identifier; forwarding the placeholder tag and streaming text data to a client; wherein the client is configured to display the streaming text data in real time on a display page, and to locate and update the placeholder element corresponding to the image position on the display page according to the placeholder tag.

[0027] Step 210: Receive streaming text data from a third-party large model. In some embodiments, step 210 may be performed by an intermediate layer or receiving module 110.

[0028] Streaming text data refers to text content pushed in real time from a third-party large model API via a streaming protocol.

[0029] A streaming protocol is a communication protocol that allows data to be sent from one end (such as a server) to another end (such as a client) in the form of a continuous data stream. Examples of streaming protocols include Server-SentEvents (SSE) protocol, WebSocket protocol, or Real-time Transport Protocol (RTP).

[0030] In some embodiments, when a user (e.g., a construction worker) asks a question through a client (e.g., a question-and-answer system), a third-party big model can receive the user's question, retrieve the answer from a local knowledge base, generate streaming text data from the relevant data used to answer the user's question, and send it to the middle layer.

[0031] A client refers to a software application used by an end user, such as a web browser or mobile application.

[0032] In some embodiments, the middleware layer receives streaming text data by establishing an SSE connection with a third-party large model. For example, when calling the API of the third-party large model, parameters can be set to enable streaming mode. The middleware layer continuously listens to this SSE connection and receives text fragments generated one by one by the third-party large model in the form of a data stream.

[0033] In some embodiments, the middleware layer can also receive data via the WebSocket protocol. In this case, the middleware layer establishes a long-lived WebSocket connection with a third-party large model and continuously processes text message frames pushed by the server in real time.

[0034] In some embodiments, the middle layer may: establish a client session corresponding to the user and assign a session identifier to the client session; determine whether the last active time of the client session meets a preset condition; and, in response to the last active time of the client session meeting the preset condition, perform a cleanup operation based on the client session.

[0035] A client session refers to a unique connection state established between a client and a server for a complete interaction. The middleware layer can create a private memory space for each user's client session to store the real-time processing state and data of that session, thus isolating data processing for different users. For example, when a user opens a question-and-answer system and begins a conversation, the system creates a client session for that user to track the context information of the conversation. A client session can include resources such as context information and network connectivity.

[0036] In some embodiments, a client session may include a sliding window buffer, a placeholder registry, a metadata cache, etc. For more information on the sliding window buffer, please refer to step 220 and its related description. For more information on the placeholder registry, please refer to step 230 and its related description.

[0037] A session identifier is a globally unique identifier assigned to each client session.

[0038] In some embodiments, when a user interacts with the question-and-answer system for the first time (e.g., by initiating a new question-and-answer request), the middleware instantiates a session object. This session object acts as a container to store and process all the state information required for that client session, such as initializing a separate sliding window buffer and a placeholder registry.

[0039] In some embodiments, the middleware assigns a unique session identifier to the client session when it is created. For example, a globally unique session identifier can be created by calling a Universally Unique Identifier (UUID) generation library. Alternatively, the session identifier can be generated by combining the user ID, the current timestamp, and a random number, and then performing a hash operation (e.g., SHA-256).

[0040] In some embodiments, the establishment of a client session and the allocation of a session identifier can be completed automatically at the beginning of the client session lifecycle, and any generation algorithm that can guarantee the uniqueness of the identifier within the system can be applied.

[0041] Last active time refers to the timestamp of the last data activity recorded in the client session, which is updated each time streaming text data or user heartbeat signals are received.

[0042] The preset condition refers to a predefined session timeout threshold used to determine whether a client session has expired.

[0043] In some embodiments, the preset condition may be that the last active time is more than a specific duration from the current time, such as 5 minutes, 10 minutes, or 15 minutes. A background scheduled task within the middle layer can periodically (e.g., every 1 minute or every 2 minutes) iterate through all active client sessions. For each client session, the scheduled task checks the last active time of that client session and calculates the difference between it and the current time. If the difference is greater than a preset timeout duration, the preset condition is considered met.

[0044] In some embodiments, whenever the middle layer receives new streaming text data from a third-party large model or receives a heartbeat packet from the client, it updates the last active time in the corresponding client session to the current time, thereby resetting the idle timer.

[0045] Cleanup operations refer to a series of resource release and connection termination actions performed on timed-out client sessions. Examples of cleanup operations include closing network connections, clearing memory data, and removing session indexes.

[0046] In some embodiments, when a client session is determined to meet preset conditions, it is considered a timeout client session, and the middleware layer triggers a cleanup operation. The middleware layer first closes all network connections associated with the client session, for example, closing SSE or WebSocket connections with the client and third-party large models. Subsequently, the middleware layer releases all memory resources occupied by the client session, including destroying the sliding window buffer, clearing the placeholder registry, and removing it from the global session management map so that the garbage collection mechanism can reclaim the memory.

[0047] In some embodiments, the cleanup operation may also include sending a session timeout notification event to the client so that the client can update the user interface accordingly, for example, prompting the user "The session has ended, please start again".

[0048] In some embodiments, the specific steps of the cleanup operation can be adjusted according to the specific architecture of the system, with the core objective of safely and thoroughly releasing all system resources associated with the failed session.

[0049] In some embodiments of this specification, establishing client sessions effectively isolates the session states of different users, ensuring the correctness of data processing. Simultaneously, through timeout checks and automatic cleanup mechanisms, valuable resources such as memory and network connections occupied by long-term inactive sessions can be released promptly, effectively preventing resource leaks and significantly improving the stability and scalability of the method in high-concurrency scenarios.

[0050] Step 220: Based on the streaming text data, resource reference tags are identified to determine the resource address corresponding to the resource reference tag. In some embodiments, step 220 may be performed by the intermediate layer or the address determination module 120.

[0051] Resource reference tags are structured tags embedded in streaming text data used to reference external resources (such as images, audio, or video). For example, in Markdown syntax, the format of a resource reference tag for referencing an image can be .

[0052] A resource address is a Uniform Resource Locator (URL) extracted from a resource reference tag, used to uniquely identify the location of a resource on a network.

[0053] In some embodiments, the intermediate layer can identify resource reference tags by combining sliding windows and regular expressions.

[0054] In some embodiments, the middleware layer may: establish a sliding window buffer corresponding to the client session, the sliding window buffer being used to cache text characters of streaming text data; match the text characters in the sliding window buffer using a preset regular expression to identify resource reference tags; and, in response to successfully identifying a resource reference tag, parse the resource address from the tag structure.

[0055] A sliding window buffer (or simply buffer) is a fixed-size memory area used to temporarily store text characters of streaming text data. In some embodiments, the size of the sliding window buffer can be set to one of 100 text characters, 200 text characters, 300 text characters, 500 text characters, etc.

[0056] In some embodiments, the sliding window buffer can be implemented using a double-ended queue (Deque) data structure. When the intermediate layer receives new text characters from a third-party large model, it can append these text characters to the tail of the queue. Simultaneously, to maintain a stable buffer size, when the buffer content exceeds a preset capacity, the intermediate layer removes older text characters from the head of the queue. For example, the size of the sliding window buffer can be set to 200 text characters, which is sufficient to accommodate the length of a complete resource reference tag, thus ensuring the integrity of the match.

[0057] In some embodiments, the sliding window buffer can also be implemented using a ring buffer. This structure efficiently adds and removes characters by maintaining read and write pointers, avoiding frequent memory reallocation, and is suitable for high-performance streaming data processing scenarios.

[0058] Preset regular expressions are predefined strings used to find and match specific patterns in text characters. Preset regular expressions are used to identify and match the syntax structure of resource reference tags.

[0059] Tag structure refers to the specific syntactic format and components that a resource reference tag follows in text. For example, the tag structure of a Markdown image tag "" includes the opening tag "![", the description text "alt text", the URL address "image.url", and the corresponding parentheses.

[0060] In some embodiments, the default regular expression can be constructed based on the Markdown image tag specification. When the streaming text data follows Markdown syntax, the default regular expression can be designed specifically to match patterns of Markdown image tags. For example, the regular expression "!([^]+)]([)]+)" can be used. This pattern can recognize standard image tags that begin with "![", end with ")", and contain a description and URL in between.

[0061] In some embodiments, the parsing process for obtaining the resource address from the tag structure utilizes the capturing group feature of regular expressions. For example, for the regular expression "!([^]+)]([)]+)", the resource address corresponds to the second group. When a match is successful, the intermediate layer directly extracts the content of the second group (i.e., the parentheses) from the matching result object to obtain the accurate resource address.

[0062] In some embodiments of this specification, a sliding window and regular expressions are used to achieve low-latency, high-efficiency identification of resource reference tags in streaming text data. This eliminates the need to wait for the entire data stream to finish, enabling real-time processing as data arrives, effectively solving the problem of tags being segmented by data blocks. Compared to full caching followed by processing, this method significantly reduces memory consumption and processing latency, ensuring a smooth user experience.

[0063] In some embodiments, to improve matching efficiency, the intermediate layer may employ an incremental matching strategy during matching. In some embodiments, the intermediate layer may: record the end position index of the previous matching operation; when a new text character is added to the sliding window buffer, perform regular expression matching based on a preset regular expression on a sub-window interval starting from the end position index and extending backward by a preset length; and in response to a successful regular expression match, update the end position index to the end position of the current match.

[0064] The end position index of the last matching operation refers to the position of the end character of the matched tag string in the buffer when the resource reference tag was most recently successfully matched.

[0065] In some embodiments, the end position index (e.g., "lastMatchEndIndex") is stored as an integer variable in each individual client session and initialized to 0. The end position index is only updated after a complete resource reference tag is successfully identified.

[0066] In some embodiments, the end position index can also be implemented as a pointer or iterator to an internal data structure of the buffer. When a match is successful, the pointer or iterator moves forward to the end of the matched portion.

[0067] New text characters refer to the latest text data fragments received in real time from the third-party large model API and appended to the buffer.

[0068] In some embodiments, when the buffer receives a new text character, the intermediate layer does not scan the entire buffer. Instead, it starts from the end position index of the record, extracts a substring of fixed length (e.g., 200 text characters, which is greater than the maximum possible length of a resource reference tag), and then performs regular expression matching only on this substring.

[0069] In some embodiments, the intermediate layer may also utilize a regular expression engine that supports specifying the starting position of the search. In this case, the intermediate layer takes the entire contents of the buffer as input, but passes the end position index of the record as a parameter to the matching function, instructing it to start the search from that end position index, thereby avoiding the overhead of creating temporary substrings.

[0070] In some embodiments, in addition to regular expression matching, other incremental analysis techniques can be used for matching. For example, a finite state automaton can be implemented to consume newly arriving text character by character, change states according to preset syntax rules, and when the final acceptance state is reached, it means that a tag has been successfully identified.

[0071] A successful match indicates that the regular expression matching operation detected a complete resource reference tag that matches the preset regular expression pattern within the sub-window range.

[0072] In some embodiments, when a regular expression successfully matches within a sub-window region, it returns an object containing matching information. The intermediate layer can extract the end position index of the matched string in the original sliding window buffer from this object and update the end position index stored in the client session with this value.

[0073] In some embodiments, if multiple matches may exist within a sub-window range, the intermediate layer is configured to perform a cyclic search. After processing the first match and updating the end position index, the matching operation immediately resumes from the new end position index, continuing to search for the next match within the current sub-window until the entire sub-window range has been scanned. If no match is found, the end position index remains unchanged.

[0074] In some embodiments, the update logic for the end position index is related to the specific matching strategy, with the core objective being to ensure that the next matching operation can seamlessly continue from the end of the processed text.

[0075] In some embodiments of this specification, incremental scanning is achieved by recording the end position of the previous match and performing regular expression matching only on the newly added text and its adjacent sub-windows. This method avoids repeated scanning of the entire sliding window buffer every time data arrives, significantly reducing redundant calculations, lowering the computational overhead and CPU utilization of regular expression matching, and ensuring low-latency identification of resource tags even in high-speed text stream scenarios, thus guaranteeing the real-time performance of data processing and the overall performance of the intermediate layer.

[0076] In some embodiments, the intermediate layer may further: after a new text character is added to the sliding window buffer, determine whether the text characters in the sliding window buffer exceed a preset capacity limit; in response to the text characters in the sliding window buffer exceeding the preset capacity limit, based on the buffering time of the text characters, remove some text characters from the sliding window buffer to release the capacity space of the sliding window buffer; and save the removed text characters to the underlying reuse array corresponding to the sliding window buffer.

[0077] The preset capacity limit refers to the maximum character capacity of the buffer defined in advance. When the number of text characters in the buffer reaches the preset capacity limit, old characters need to be removed to make room for new characters. For example, the preset capacity limit can usually be set to 200 characters or more to ensure that the length of a resource reference tag is fully covered.

[0078] In some embodiments, the judgment operation (determining whether the number of text characters in the buffer exceeds a preset capacity limit) is performed immediately after each new text character is appended to the buffer. For example, the intermediate layer can obtain the current total number of characters by calling the length or size detection method of the buffer's data structure itself, and compare the current total number of characters with the preset capacity limit.

[0079] In some embodiments, the judgment operation can also be performed before appending text characters. For example, the intermediate layer can pre-calculate the expected total length after appending the new characters. If the expected total length exceeds a preset capacity limit, a removal operation is performed first to make room for the new characters, and then the append operation is performed.

[0080] In some embodiments, the specific timing and method of determining the operation can be adjusted according to system performance requirements, with the core objective being to ensure that the capacity of the buffer is always kept within a preset limit.

[0081] Partial text characters refer to one or more characters selected for removal from the sliding window buffer. These are usually the earliest characters added to the buffer, following the First-In, First-Out (FIFO) principle.

[0082] In some embodiments, if the buffer is implemented based on a deque, the removal operation is performed by continuously popping characters from the head of the queue (i.e., the end where the character was first added). For example, the intermediate layer calculates the number of characters exceeding a preset capacity limit and then calls the deque.pollFirst() method a corresponding number of times until the total number of characters in the buffer falls back below the preset capacity limit.

[0083] In some embodiments, if the buffer is implemented based on a ring buffer, the removal operation is performed by moving the start pointer of the buffer. For example, the intermediate layer can move the start pointer forward by a corresponding number of positions based on the number of characters exceeding a preset capacity limit. The characters that are skipped are logically considered to have been removed, and the space they occupy can be overwritten by new characters written subsequently.

[0084] The goal of the removal operation is to maintain a fixed size for the sliding window; any mechanism that can efficiently remove the oldest data to free up space is applicable. The number of characters removed is determined by freeing up enough space in the sliding window buffer to make it usable.

[0085] The underlying reuse array refers to a fixed-size array or memory region that the buffer relies on to store removed characters in order to achieve memory reuse and avoid frequent memory allocation and release.

[0086] In some embodiments, when a fixed-size array is used as the underlying storage for the circular buffer, the "remove" operation simply moves the starting pointer; the removed character data actually remains in the array's memory until a new character is written later. This mechanism naturally achieves memory reuse and avoids the overhead of garbage collection.

[0087] In some embodiments, if the intermediate layer uses object pooling to manage character objects, then character objects removed from the buffer are returned to the object pool instead of being immediately destroyed. When new character objects are needed later, they can be directly retrieved from the object pool and reused, which also falls under the category of "saving" removed characters for reuse.

[0088] In some embodiments of this specification, by setting a capacity limit for the sliding window and dynamically removing old characters that exceed the limit, unlimited memory growth during long sessions is effectively prevented, ensuring stable system operation. Performance is optimized through memory reuse technology, avoiding system overhead and latency caused by frequent memory allocation and deallocation operations. Simultaneously, by storing removed characters in an underlying reuse array (such as using a circular buffer), frequent memory allocation and reclamation operations are avoided, significantly reducing garbage collection (GC) pressure and CPU overhead, thereby ensuring low latency and high throughput performance for streaming data processing.

[0089] In some embodiments, the identification and address determination of resource reference tags can also be achieved through other techniques. For example, a state machine-based parsing method can be used to analyze the text stream character by character according to predefined syntax rules in order to identify the tag structure and extract the address.

[0090] Step 230: Generate a placeholder identifier corresponding to the resource address and replace the resource reference tag with the placeholder tag. In some embodiments, step 230 may be performed by an intermediate layer or the identifier generation module 130.

[0091] Placeholder identifiers are globally unique identifiers generated for each identified resource address, used to track the resource in subsequent processing.

[0092] In some embodiments, the middleware layer creates a globally unique identifier by calling a UUID generation library. For example, the uuid.v4() function can be called to generate a version 4 UUID as a placeholder identifier.

[0093] In some embodiments, the intermediate layer can also generate placeholder identifiers by performing a hash operation (e.g., SHA-256 algorithm) on the resource address and the current session identifier to ensure that the same placeholder identifier is always generated for the same resource address in the same client session, which facilitates deduplication or caching.

[0094] In some embodiments, the placeholder identifier can also be generated in other ways. For example, a distributed ID generation algorithm (such as the Snowflake algorithm) or an auto-incrementing counter maintained within the session can be used to generate a unique identifier.

[0095] In some embodiments, the intermediate layer is also configured to replace resource reference tags with placeholder tags.

[0096] Placeholder tags are temporary markers used to replace the original resource reference tags. The format of placeholder tags is similar to that of resource reference tags, but the resource address portion is replaced with the placeholder identifier.

[0097] In some embodiments, for a Markdown-formatted original resource reference tag, such as "", the intermediate layer constructs a new placeholder tag. For example, this placeholder tag could be "", where "Loading..." is the placeholder text, "placeholder:" is a custom protocol for client identification, and "uuid-generated-for-image" is the placeholder identifier generated in the previous step. Subsequently, the intermediate layer performs a string replacement operation on the streaming text data, replacing the original resource reference tag with this newly generated placeholder tag.

[0098] In some embodiments, the middleware layer may also: asynchronously initiate a metadata query request for the resource address to obtain local resource chapter information bound to the resource address according to the metadata mapping table; in response to finding local resource chapter information bound to the resource address, generate a first backfill event and send the first backfill event to the client so that the client can replace the placeholder element with an image hyperlink including chapter jump based on the first backfill event; the first backfill event includes a placeholder identifier, the resource address, and the local resource chapter information; in response to not finding local resource chapter information bound to the resource address, generate a second backfill event so that the client can replace the placeholder element with an image carrying no metadata tag based on the second backfill event; the second backfill event includes no metadata tag and is sent to the client.

[0099] A metadata query request is an asynchronous data retrieval operation initiated from the metadata mapping table, designed to query the bound local resource chapter information based on the resource address.

[0100] A metadata mapping table is a pre-built in-memory hash table that stores the mapping relationship between resource addresses and local resource chapter information. In some embodiments, the metadata mapping table can be pre-loaded into the memory of the middleware service process and stored using efficient data structures such as hash tables or dictionaries. The middleware uses the resource address as the key to directly query in memory, achieving millisecond-level response times.

[0101] Local resource chapter information refers to the chapter metadata of the standardized document in the local knowledge base corresponding to the resource address, including document name, chapter number, page number, etc.

[0102] In some embodiments, once a resource address is identified, the middleware layer can submit a metadata query request to a separate asynchronous task queue, and the query task can be executed by a metadata query worker thread. The middleware layer can then execute tasks from this asynchronous task queue, i.e., perform the actual query operation, thereby avoiding blocking the forwarding of streaming text data.

[0103] In some embodiments, the middle layer may also: submit a metadata query request to an asynchronous task queue, the metadata query request including a resource address and a placeholder identifier; and execute a query task from the asynchronous task queue through a metadata query worker thread independent of the text stream forwarding thread, including: querying a metadata mapping table preloaded into an in-memory database using the resource address as the key, and obtaining local resource chapter information, the local resource chapter information including the specification document name, chapter number, and page number information corresponding to the resource address.

[0104] An asynchronous task queue is a first-in, first-out (FIFO) data structure used to store metadata query tasks.

[0105] In some embodiments, when the main thread responsible for forwarding streaming text data identifies a resource reference tag, it creates a query task containing the resource address and placeholder identifiers. The main thread then places this query task into a memory-based blocking queue (e.g., LinkedBlockingQueue in Java). This delivery operation is non-blocking or has extremely low latency, allowing the main thread to immediately continue processing and forwarding subsequent text data without waiting for the query result.

[0106] In some embodiments, the asynchronous task queue can be implemented using an external message queue middleware, such as RabbitMQ or Kafka. In this case, the main thread acts as a producer, serializing metadata query requests and sending them as a message to a specified queue or topic. This approach enhances the decoupling and scalability of the system, allowing the query service to be deployed and scaled independently.

[0107] The implementation of asynchronous task queues is not limited to this; any data structure or middleware that can implement asynchronous buffering and first-in-first-out processing of tasks can be applied.

[0108] Metadata query worker threads (or simply worker threads) are dedicated threads independent of text stream forwarding threads, used to consume query tasks from the asynchronous task queue and execute query operations. In-memory databases refer to database systems that preload metadata mapping tables into memory.

[0109] In some embodiments, the middleware layer initializes a fixed-size thread pool as a metadata query worker thread pool upon startup. The worker threads in this pool continuously and blockingly listen to the asynchronous task queue. Once a new query task is available in the queue, an idle worker thread acquires the task and begins execution. This independent thread pool design ensures that the computational overhead of metadata queries does not affect the main thread's real-time forwarding of the text stream.

[0110] In some embodiments, after a worker thread retrieves a query task from the task queue, it uses the resource address contained in the query task as the key to query the metadata mapping table preloaded into an in-memory database (e.g., Redis). For example, the worker thread might send a GET or HGET command to the Redis server. Because the data is entirely in memory, the query operation typically completes within milliseconds. Upon successful query, the worker thread retrieves local resource chapter information containing the specification document name, chapter number, and page number.

[0111] In some embodiments, the metadata mapping table can also exist as a local cache within the application (e.g., ConcurrentHashMap). Worker threads perform hash lookups directly in the process's memory, which is faster but scalability is limited by the memory of a single application. The implementation of the metadata query worker thread is not limited to thread pools; it can also be a non-blocking I / O model based on an event loop. Similarly, the in-memory database can also be other types of caching systems, such as Memcached.

[0112] In some embodiments of this specification, by submitting metadata query tasks to an asynchronous task queue and processing them by an independent metadata query worker thread, the query operation and text stream forwarding are completely decoupled. This mechanism avoids time-consuming query operations blocking the main thread, ensuring the real-time performance and smoothness of streaming text. Simultaneously, by querying a mapping table preloaded into the in-memory database, the efficiency of metadata retrieval is ensured, providing a guarantee for low-latency backfilling operations and significantly improving the system's concurrent processing capabilities and user experience.

[0113] In some embodiments, the metadata mapping table is constructed by scanning and parsing the specification documents in the local knowledge base, establishing a mapping relationship between each image resource address and the chapter information of the source document; persisting the mapping relationship in a relational database and loading it into an in-memory hash table to construct the metadata mapping table.

[0114] A local knowledge base refers to a collection of standardized documents stored within an enterprise, including digital resources such as building codes and atlases, typically stored in PDF or structured data formats.

[0115] Standard documents refer to electronic documents of industry standards or authoritative guidelines, such as national standard atlases or design specifications.

[0116] In some embodiments, the middleware layer uses an automated program to scan and parse specification documents. For example, this automated program can iterate through all PDF specification documents stored in a local knowledge base and extract image resource addresses and corresponding chapter information. For each specification document, the automated program parses it page by page, extracting embedded image resources and obtaining their corresponding resource addresses. Simultaneously, the automated program parses the text content near the images, using regular expressions or location-based heuristics to identify the corresponding document name, chapter title, and page number, thereby establishing a mapping between image resource addresses and chapter information.

[0117] In some embodiments, persistent storage is accomplished by writing the established mapping relationships to a relational database (such as MySQL or PostgreSQL). For example, the middleware layer can be designed with a data table containing fields such as "image resource address," "document name," "chapter number," and "page number." After the scan and parsing are complete, the automated program iterates through all mapping relationships and executes an SQL INSERT statement for each relationship, saving it to the database to ensure long-term data availability and maintainability.

[0118] In some embodiments, the loading of the in-memory hash table is performed when the middleware service starts. For example, during the initialization phase, the middleware connects to a relational database and executes a SELECT query to retrieve all mapping relationship data. Subsequently, the middleware iterates through the query result set and loads the data into an in-memory hash table structure (such as a ConcurrentHashMap in Java or a Hash Set in Redis). In this hash table, image resource addresses serve as keys, while objects containing document names, chapter information, etc., serve as values.

[0119] In some embodiments, the persistent storage medium is not limited to relational databases, but can also be a NoSQL database (such as MongoDB) or a key-value store system. Similarly, the loaded memory structure is not limited to a local hash table, but can be a distributed caching system (such as Memcached) to support horizontal scaling.

[0120] In some embodiments of this specification, an accurate and maintainable metadata mapping table is constructed through automated scanning, parsing, and persistent storage. This mapping table is preloaded into an in-memory hash table, providing millisecond-level response capabilities for runtime metadata queries. This construction method separates time-consuming data preprocessing from real-time online services, ensuring both the reliability and consistency of the data source and the high performance of online queries, thus laying the foundation for the low-latency backfilling mechanism of the entire system.

[0121] In some embodiments, the middle layer may: determine a metadata query strategy based on the resource type and client session corresponding to the resource reference tag; allocate metadata query requests to the corresponding query execution path based on the metadata query strategy, wherein the query execution path includes at least one of a fast path, a standard path, and a complex path; and query the local resource chapter information bound to the resource address based on the query execution path.

[0122] Metadata query strategy refers to the query rules or configurations determined based on resource type and client session, which guide the subsequent query execution method.

[0123] Resource type refers to the category of resource inferred from the resource address. For example, resource type can be "internal knowledge base image", "external public image", or "dynamically generated chart".

[0124] In some embodiments, the metadata query strategy is determined by a pre-defined rule engine. After identifying a resource reference tag, the middleware first parses the resource address to infer its resource type (e.g., distinguishing between "internal" and "external" by determining whether its domain name is an internal CDN domain). Simultaneously, the middleware reads relevant information from the context of the current client session (e.g., user permission level, current network status, etc.). Subsequently, the rule engine matches predefined rules based on these inputs (e.g., resource type "internal," user permission "advanced") to output a defined metadata query strategy (e.g., a "fast query strategy").

[0125] In some embodiments, the determination of the metadata query strategy can be dynamically provided by a separate configuration service. The middle tier sends key information about the resource type and client session to the configuration service, which returns the most suitable query strategy based on global policies or real-time system load. This approach allows for policy adjustments without redeploying the main service, improving flexibility.

[0126] In some embodiments, the method for determining the metadata query strategy is not limited to this. For example, a trained machine learning model can be used to predict the optimal query strategy based on various characteristics of the resource and historical query performance.

[0127] The query execution path refers to the different processing channels to which a metadata query task is routed, and each channel has specific performance characteristics and resource usage rules.

[0128] A fast path refers to an execution path designed for resources whose metadata has been preloaded into an in-memory database, used to perform direct key-value queries.

[0129] The standard path refers to the execution path designed for resources that need to access external APIs or perform lightweight computing, and is typically assigned to a task queue with standard priority.

[0130] Complex paths refer to execution paths designed for resources that require heavy content analysis or computation, and are typically assigned to low-priority background task queues.

[0131] In some embodiments, the allocation process is executed based on the identifier of the metadata query strategy. For example, if the determined strategy is a "fast query strategy," the middleware layer executes the query task directly in the current thread or a high-priority thread pool, corresponding to the fast path. If the determined strategy is a "standard query strategy," the middleware layer encapsulates the query task and submits it to a standard-priority asynchronous task queue, which is then executed by the consumer thread of that queue, corresponding to the standard path. If the determined strategy is a "complex query strategy," the query task is submitted to a separate, low-priority task queue with lower concurrency, corresponding to the complex path.

[0132] In some embodiments, the metadata query policy itself can be an object containing detailed configuration, rather than a simple identifier. This object explicitly specifies the name of the target task queue, the task priority, and the timeout. The allocation module parses this policy object and routes the query request precisely to the specified execution path based on its configuration.

[0133] In some embodiments, the allocation of query execution paths can also be implemented in other ways. For example, the middleware layer can be configured with a dynamic routing gateway, which adaptively allocates tasks based on the real-time load of the processing queues of each execution path to ensure overall stability and response speed.

[0134] In some embodiments, for query requests assigned to a fast path, the worker thread executing the query directly searches in an in-memory database (such as Redis) or a local cache (such as ConcurrentHashMap) using the resource address as the key, and immediately returns the local resource chapter information.

[0135] In some embodiments, for a query request assigned to a standard path, after the worker thread retrieves the query task from the asynchronous task queue, it may execute an external HTTP request to invoke a metadata service, or execute a lightweight script to extract the required information from the raw data.

[0136] In some embodiments, for query requests assigned to complex paths, worker threads may perform a time-consuming operation, such as downloading the entire resource file, running optical character recognition (OCR) or image analysis algorithms on its contents to dynamically generate metadata, and then returning the chapter information.

[0137] In some embodiments, the specific implementation of the query is closely related to the design of the execution path. For example, a query task may first try to look in the cache of the fast path, and if it is not found, it will then fall back to the standard path to query through the API, forming a multi-level query strategy.

[0138] In some embodiments of this specification, refined and differentiated management of metadata queries is achieved by determining query strategies and allocating requests to different query execution paths. This method provides fast paths for high-frequency, critical queries to ensure low-latency responses, while isolating time-consuming and complex queries to low-priority paths to prevent them from impacting core system performance. This significantly improves system resource utilization efficiency and stability under high load, ensuring a superior user experience across various scenarios.

[0139] In some embodiments, the middleware layer may: acquire the retrieval enhancement expression generated based on the current session with the user before receiving streaming text data from a third-party large model; predict the set of resource addresses that may be referenced in the streaming text data based on the retrieval enhancement expression; asynchronously initiate a metadata query request for the resource address and preload the query result into the session cache; determine whether the resource address belongs to the resource address set, and if the resource address belongs to the resource address set, directly retrieve the local resource chapter information bound to the resource address from the session cache.

[0140] Retrieval-Augmented Expressions (RAGs) refer to relevant document fragments and resource information retrieved from a local knowledge base after a user submits a query, using Retrieval-Augmented Generation (RAG) technology. These relevant document fragments and resource information are integrated into structured data to construct input prompts for third-party large-scale application programming interfaces (APIs). Retrieval-Augmented Expressions include textual descriptions and resource references, aiming to enhance the accuracy and relevance of AI responses.

[0141] In some embodiments, when a user submits a question, the middleware layer first performs a RAG search, using vector retrieval technology to find the most relevant text snippets in the vector index of a local knowledge base (e.g., a building code document library). These text snippets may contain text descriptions and resource references such as images. The middleware layer combines these retrieved contents to form a context-rich enhanced search expression. For example, the enhanced search expression includes the text snippet "The length of the stirrup reinforcement zone at the beam end is twice the beam height" and the image resource reference "..." ".

[0142] In some embodiments, the process of obtaining the enhanced search expression can employ a hybrid search strategy. For example, the intermediate layer can combine keyword-based sparse vector search and semantic-based dense vector search to improve the accuracy and coverage of search results. Finally, all retrieved relevant information is integrated into a single enhanced search expression.

[0143] In some embodiments, the enhanced retrieval expression can also be generated using other information retrieval techniques, such as constructing context by querying a knowledge graph to extract entities and relationships relevant to the user's question.

[0144] The resource address set refers to the set of Uniform Resource Locators (URLs) for resources such as images extracted from search-enhanced expressions.

[0145] In some embodiments, the set of resource addresses is predicted by parsing the text content of the retrieved enhanced expression. For example, the intermediate layer may apply a preset regular expression (e.g., matching the src="URL" pattern) or a lightweight HTML parser to scan the retrieved enhanced expression, identify and extract all resource reference tags (such as...) The resource addresses of the tags are deduplicated and stored in a set to obtain the resource address set.

[0146] In some embodiments, if the retrieved augmentation expression itself is structured (e.g., in JSON format), it may contain a field specifically for storing resource references (e.g., an array of retrieved_resources.images). In this case, the prediction process involves directly reading from this field and collecting all resource addresses.

[0147] In some embodiments, the intermediate layer may be trained to a natural language processing model for identifying and extracting potentially referenced resource entities and their addresses from unstructured text.

[0148] Session caching refers to a memory storage area set up in the session context to temporarily store metadata query results in order to speed up subsequent access.

[0149] In some embodiments, the middleware layer can treat the entire set of resource addresses as a batch task and submit it to an asynchronous task queue. One or more background worker threads consume the batch task, iterate through each resource address in the set, query its corresponding metadata, and store the query results (e.g., key-value pairs with resource address as key and metadata as value) in the session cache of the current session.

[0150] In some embodiments, to further improve preloading efficiency, the middleware layer can create an independent asynchronous query task for each address in the resource address set and execute these asynchronous query tasks concurrently using a thread pool. The query results of all asynchronous query tasks are concurrently written to the same thread-safe session cache.

[0151] In some embodiments, when a resource address is identified, the intermediate layer first checks whether the resource address exists in a pre-generated set of resource addresses. For example, this can be achieved by performing a lookup operation in a hash set, which has an average complexity of near constant time (O(1)).

[0152] In some embodiments, in response to the resource address belonging to the resource address set (i.e., a cache hit), the middleware layer uses the resource address as the key to directly retrieve the pre-loaded local resource chapter information from the session cache (e.g., a hash table). This retrieval process is almost instantaneous, avoiding the initiation of new, delayed database or network queries. The retrieved local resource chapter information is then used to generate a backfill event and sent to the client. If the result is negative (i.e., a cache miss), the middleware layer falls back to the standard on-demand asynchronous query process.

[0153] In some embodiments of this specification, a proactive "prediction-preloading-instant backfilling" model is used to remove metadata queries from the real-time serial path. This method asynchronously completes the query and stores the results in a session cache before the AI ​​generates an answer. When a resource reference appears in the streaming text data, the information can be instantly retrieved from the cache. This virtually eliminates the latency introduced by metadata queries, resolves the core contradiction between streaming transmission and time-consuming queries, and greatly improves the system's smoothness and responsiveness.

[0154] The first backfill event is the event generated when a query hits. The first backfill event includes the placeholder identifier, resource address, and local resource section information, used to notify the client to update the placeholder element.

[0155] Image hyperlinks that include chapter navigation are interactive image elements configured to navigate users to a specific chapter or location within a related document when triggered by the user. For example, an image hyperlink that includes chapter navigation could be a construction detail displayed in a Q&A interface. When a user clicks on the detail, the client automatically opens an online specification document viewer and directly navigates to the specific chapter and page containing the detail.

[0156] In some embodiments, after the intermediate layer successfully retrieves the local resource chapter information, a first backfill event is constructed. This first backfill event includes a placeholder identifier, the actual resource address, and information such as the document name, chapter number, and page number obtained from the metadata mapping table.

[0157] In some embodiments, the first backfill event may also be in JSON format, XML format, or other custom binary format to meet performance or compatibility requirements in different scenarios.

[0158] In some embodiments, the generated first backfill event is pushed to the client via the SSE protocol. The middleware layer sends a specific type of event (e.g., metadata_ready) over the SSE connection established with the client, the data payload of which is the generated first backfill event.

[0159] In some embodiments, if the middleware layer communicates with the client using the WebSocket protocol, the first backfill event is sent to the client as a separate message frame via the WebSocket connection. The method of sending the first backfill event is not limited to SSE or WebSocket; other server push technologies can also be used.

[0160] The second backfill event is generated when a query fails to find a match. The second backfill event includes a placeholder identifier, a resource address, and a "no metadata" tag, used to notify the client to downgrade and update the placeholder element.

[0161] A "no metadata" tag is an information marker used to indicate that no associated metadata was found for a specific resource. For example, a "no metadata" tag could be a boolean flag (such as "fallback": true) included in a fallback event, used to notify the client that the image corresponding to the resource address did not find associated specification section information in the local knowledge base, and therefore the client should only display the image corresponding to that resource address without adding a jumpable section link.

[0162] Degradation refers to automatically switching to a simplified backup processing mode when full or optimal functionality cannot be provided, in order to ensure the availability of core services and a basic user experience. For example, if the middleware layer fails to find the relevant specification document section information for an image, degradation will be triggered, thus only showing the image itself to the user without providing a clickable hyperlink to the specification document.

[0163] In some embodiments, when a query operation does not find a match in the metadata mapping table, the middleware layer generates a second backfill event. The second backfill event also contains a placeholder identifier and a resource address, but additionally includes a "no metadata" tag, and the local resource section information field is empty. When the client receives the second backfill event from the middleware layer, it replaces the placeholder element with an image carrying the "no metadata" tag based on the second backfill event. The resource address and placeholder identifier in the second backfill event indicate the information the client needs to process upon receiving the second backfill event, and can be sent separately to the client.

[0164] In some embodiments, the second backfill event may also be in JSON format, XML format, or other custom binary format to meet performance or compatibility requirements in different scenarios.

[0165] In some embodiments, when a metadata query operation fails due to a network timeout or database error, the middleware will also generate a second backfill event to notify the client to perform a degradation process.

[0166] In some embodiments, the second backfill event is sent in the same way as the first backfill event, for example, by sending it as a metadata_ready event type via an SSE connection, or by sending it via WebSocket.

[0167] In some embodiments, the second fallback event can also be defined as a separate event type (e.g., metadata_fallback) so that the client can differentiate between them.

[0168] The methods provided in some embodiments of this specification decouple time-consuming I / O operations from the main text stream processing by asynchronously initiating metadata queries, ensuring the smoothness of text data streaming forwarding and avoiding user-perceived lag. By designing two types of backfill events, both successful and failed queries can be handled elegantly. In successful queries, image hyperlinks enhance information traceability and interactivity; in failed queries, a smooth degradation mechanism ensures system robustness and consistent user experience.

[0169] In some embodiments, the middleware layer may: after generating a placeholder identifier, create a session record in a pre-generated placeholder registry, the session record being used to track the query status, resource address, and metadata of the placeholder identifier; create a scheduled task, the scheduled task being used to detect the placeholder registry; in response to detecting in the placeholder registry that a placeholder identifier has not completed the metadata query within a preset time, mark the placeholder identifier status as timed out and send a downgrade backfill event to the client; in response to detecting in the placeholder registry that a placeholder identifier has completed backfilling or has been marked as timed out, remove the placeholder identifier that has completed backfilling or has been marked as timed out from the placeholder registry after a preset delay time.

[0170] A placeholder registry is a collection of records used to store and track the processing status of placeholders. For example, a placeholder registry can contain multiple entries, each corresponding to a placeholder identifier, and recording information such as the original resource address corresponding to that identifier, the current metadata query status (e.g., 'in progress,' 'completed,' or 'timeout'), and the creation time.

[0171] Session records refer to the data entries created for each placeholder identifier in the placeholder registry, used to track the entire processing lifecycle of the placeholder. Session records include fields such as the query status, resource address, and metadata of the placeholder identifier.

[0172] Query status refers to the status indicator in the session record that represents the progress of placeholder metadata query, and is used to manage the processing flow of placeholders. Query status includes initial status, in progress status, completed status, and abnormal status.

[0173] Metadata refers to the local resource chapter information corresponding to the resource address, including the standard document name, chapter number, page number, etc.

[0174] In some embodiments, once the middleware layer recognizes a resource reference tag and generates a placeholder identifier, it immediately creates a session record in the placeholder registry. For example, this placeholder registry can be an in-memory concurrent hash table (such as a ConcurrentHashMap in Java), using the placeholder identifier as the key and an object containing information such as status, resource address, and creation timestamp as the value. Upon initial creation, the session record's query status is set to "PENDING".

[0175] In some embodiments, the placeholder registry can be implemented by an external key-value store system (such as Redis). The middleware layer can use the HSET command to create a hash in Redis with the placeholder identifier as the key, containing fields such as "status" and "original_url". This approach supports state tracking in a distributed environment.

[0176] In some embodiments, the placeholder registry can also be implemented using other data structures or databases, such as in-memory databases (like H2) or NoSQL databases, with the core purpose of providing efficient read, write, and tracking of placeholder lifecycle states.

[0177] Scheduled tasks refer to background tasks that periodically execute detection operations in the middleware service to check the status and processing progress of all session records in the placeholder registry.

[0178] In some embodiments, the middleware layer can create scheduled tasks using the scheduler built into the programming framework. For example, Java's ScheduledExecutorService can be used to submit a scheduled task when the middleware service starts. This scheduled task is configured to execute at a fixed frequency (e.g., every 5 seconds), and its task content is to iterate through all session records in the placeholder registry.

[0179] In some embodiments, for distributed systems, a separate distributed task scheduling framework (such as Quartz or Celery) can be used to create and manage scheduled tasks. This allows tasks to execute on any node in the cluster and provides greater reliability and manageability.

[0180] The preset timeout refers to a predefined timeout threshold used to determine whether a metadata query identified by a placeholder has timed out. For example, the preset timeout could be set to 5 seconds based on network latency and query performance.

[0181] Timeout means that the placeholder identifier is marked as abnormal because the metadata query was not completed within the preset time.

[0182] The degradation backfill event is an event sent by the client to notify the client that the placeholder metadata query has timed out and the client needs to perform degradation rendering. The degradation backfill event includes a placeholder identifier, a degradation flag, and a prompt message.

[0183] In some embodiments, when the scheduled task iterates through the placeholder registry, it checks each session record whose query status is still "in progress". By comparing the current time with the session record's creation timestamp, if the time difference exceeds a preset time, the query status of that session record is updated to "timeout". Subsequently, the middleware constructs a fallback event containing the placeholder identifier and a fallback flag (such as "fallback": true) and sends it to the client via an established SSE or WebSocket connection.

[0184] In some embodiments, the middleware layer can set different preset times based on the type of query. For example, for metadata queries that require complex calculations, the preset time can be set longer than for simple in-memory queries to achieve more granular timeout management.

[0185] In some embodiments, in addition to sending degradation backfill events to the client, the middleware layer can also record timeout logs for subsequent performance analysis and troubleshooting.

[0186] In some embodiments, the scheduled task also checks session records with a status of "COMPLETED" or "TIMEOUT" during scanning. For these session records, the scheduled task checks their status update timestamp. If the difference between the current time and the update timestamp exceeds a preset delay (e.g., 30 seconds), the session record can be deleted from the placeholder registry from the intermediate layer. This delay ensures that the client has sufficient time to process the final backfill event, preventing session records from being prematurely deleted.

[0187] In some embodiments, if the placeholder registry is based on a system that supports Time-To-Live (TTL) records (such as Redis), the cleanup operation can be automated. When a placeholder's status is updated to a final state ("Completed" or "Timeout"), the intermediate layer can set a TTL for that session record. Redis automatically deletes it after the preset time, simplifying the cleanup logic. The purpose of the cleanup operation is to reclaim memory resources; any mechanism that can safely remove processed or expired records is applicable.

[0188] In some embodiments of this specification, a placeholder registry and scheduled tasks are used to achieve comprehensive management of the placeholder lifecycle. This method, through timeout detection and degradation-based backfilling mechanisms, ensures timely feedback to users even in cases of abnormal metadata queries or excessively long processing times, guaranteeing a continuous user experience. Simultaneously, by periodically cleaning up completed or timed-out records, memory leaks are effectively prevented, significantly improving the stability and reliability of the system under high concurrency and long-term operation scenarios.

[0189] Step 240 involves forwarding the placeholder tags and streaming text data to the client, which is configured to display the streaming text data in real time on the display page, and to locate and update the placeholder element corresponding to the image position on the display page based on the placeholder tags. In some embodiments, step 240 may be performed by the middleware layer or the forwarding module 140.

[0190] In some embodiments, the middleware layer pushes placeholder tags and streaming text data to the client via the SSE protocol. The middleware layer encapsulates fragments of streaming text data containing placeholder tags into SSE events and continuously pushes them through the SSE connection established with the client.

[0191] In some embodiments, the middleware layer can also implement data forwarding via the WebSocket protocol. A full-duplex WebSocket long connection is established between the middleware layer and the client, through which the middleware layer sends text data and placeholder tags to the client in real time in the form of message frames.

[0192] In some embodiments, the forwarding protocol is not limited to SSE or WebSocket. For example, HTTP long polling or gRPC streaming technologies can be used to achieve real-time data forwarding in specific scenarios.

[0193] The display page refers to the user interface component in the client used to display streaming text data in real time, such as a chat window or message panel.

[0194] Placeholder elements refer to Document Object Model (DOM) elements rendered based on placeholder tags on a display page. Placeholder elements can be represented as a placeholder area (such as a loading animation icon).

[0195] In some embodiments, after receiving streaming text data, the client uses a streaming Markdown parsing library or custom rendering logic to append the text content to the display page in real time. When the client parses a placeholder tag, it renders a placeholder element at the corresponding position. For example, it can render a placeholder element with a loading animation. The `<placeholder>` tag is used to store the placeholder identifier (e.g., "placeholder:uuid-xxx") in a custom data attribute of the DOM element, such as "data-placeholder-id='uuid-xxx'". This makes it easy to find the element later using this identifier.

[0196] In some embodiments, the client can listen to a separate event channel (e.g., different event types on the same SSE connection, or a dedicated WebSocket message channel) to receive backfill events (e.g., a first backfill event or a second backfill event) from the middleware. Upon receiving a backfill event containing a resource address and a placeholder identifier, the client locates the corresponding placeholder element on the display page by querying custom data attributes based on the placeholder identifier. Subsequently, the client updates the src attribute of the placeholder element to the actual resource address and can add other information, such as jump links, based on additional information (e.g., chapter links) in the backfill event, thereby completing the final display of the image.

[0197] In some embodiments, after locating the placeholder element, the client can update it based on the content of the backfill event. For example, if the backfill event indicates that the metadata query was successful (i.e., the local resource chapter information bound to the resource address was found), the client updates the image source (src attribute) of the placeholder element to the actual resource address and dynamically wraps a hyperlink around the placeholder element. The image tag points to the document section associated with the resource, thus enabling users to jump to the desired page when the image is clicked.

[0198] In some embodiments, the client may also forgo using hyperlink tags and instead add a custom link data attribute (such as data-chapter-link) to the placeholder element, and bind a click event listener to the placeholder element. When the user clicks the placeholder element, the event listener reads the link data attribute and programmatically opens a new page, facilitating the execution of additional logic such as event tracking or permission verification before the page redirects.

[0199] In some embodiments, the client is also configured to handle degradation scenarios. If the received backfill event indicates that the metadata query failed or timed out (i.e., no local resource chapter information bound to the resource address was found), the client may only update the placeholder element to the real image (if the resource address is available), but will not add a jump link and may display a "No source information available" message below the image.

[0200] In some embodiments, the client can provide a feedback entry point for users in degraded scenarios. For example, a "Feedback" button or link can be added next to the "No source information available" prompt, allowing users to input feedback information. After the user submits feedback information, the client can send the feedback information to the server for storage. Staff will periodically review the stored feedback information and supplement the knowledge base with missing metadata to gradually make up for the image annotations missed during the automated data cleaning process, forming a closed loop for continuous improvement of knowledge base quality.

[0201] In some embodiments, after each backfill event is processed, the client can record the performance metrics of the backfill event. These performance metrics can help to identify problems in a timely manner (such as sudden increases in metadata query latency, image CDN failures, etc.) and enable staff to quickly intervene and handle them for the optimization of the server monitoring system.

[0202] In some embodiments, when a user clicks on an image that has been successfully linked with a hyperlink, the client can navigate to the enterprise's internal online specification document viewer (PDF Viewer). After successful verification by the online viewer, the client can load the PDF document and perform operations such as page navigation, image highlighting, and chapter title hints. This allows users to seamlessly jump to the precise location in the specification document after clicking on the image in the answer, quickly obtaining complete and authoritative information, significantly improving the efficiency of consulting specifications on construction sites. When a user clicks on an image to jump to the online specification document viewer, the client can also send behavior logs and store them in a data analysis system (such as Elasticsearch or ClickHouse) to analyze the user's specification viewing habits and frequently accessed chapters. Behavior logs may include user identity, session identifier, placeholder elements, the linked chapter, the user's original question, and click timestamps.

[0203] As an example, when a technician at a construction site uses a client application integrated with the embodiments of this specification to ask a question about "the length of the stirrup reinforcement zone at the end of a frame beam," the system backend (server) retrieves a knowledge fragment containing an image reference through RAG retrieval and sends it along with the question to a third-party large model. The middle layer receives the streaming text data generated by the large model. When an image-type resource reference tag appears in the streaming text data, the middle layer identifies the resource reference tag, immediately replaces it with a placeholder tag, and pushes the text containing the placeholder tag to the client. The client displays the text in real time and shows a loading animation at the image location. Simultaneously, the middle layer asynchronously queries the image's metadata (such as its specific chapter and page number in the "16G101-1 Atlas") and sends the query results to the client as a backfill event. Upon receiving the backfill event, the client replaces the loading animation with the actual image and adds a hyperlink. After clicking the image, the technician can directly jump to the exact page in the standard atlas where the image is located and see the relevant content highlighted, thus quickly and accurately resolving the problem.

[0204] In some embodiments of this specification, by deploying an intermediate layer and employing a "placeholder-backfilling" mechanism, decoupling text content from resource processing is achieved without modifying the third-party large model interface or disrupting the streaming experience. This method resolves the text stream display lag issue caused by synchronous resource loading, ensuring smooth user interaction. Simultaneously, it can deliver the deep correlation information between resources in the local knowledge base and the chapters of the specification document to the user in real time, significantly improving the efficiency of information tracing and the interactive experience.

[0205] It should be noted that the above description of process 200 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to process 200 under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0206] This specification provides an intermediate layer device for streaming data processing, deployed between a server and a third-party large model. The intermediate layer device includes: a receiving layer configured to receive streaming text data from the third-party large model; a parsing layer configured to identify resource reference tags based on the streaming text data, determine resource addresses corresponding to the resource reference tags, generate placeholder identifiers corresponding to the resource addresses, and replace the resource reference tags with placeholder tags, the placeholder tags including the placeholder identifiers; and a forwarding layer configured to forward the placeholder tags and the streaming text data to a client. The client is configured to display the streaming text data in real time on a display page, and to locate and update placeholder elements corresponding to image positions on the display page based on the placeholder tags.

[0207] In some embodiments, the receiving layer, parsing layer, and forwarding layer are independent sub-devices, and the receiving layer, parsing layer, and forwarding layer communicate in a decoupled manner through message queues.

[0208] Independent sub-devices can be deployed on separate computing devices, such as servers. In some embodiments, the receiving layer, parsing layer, and forwarding layer can be deployed as independent microservices. For example, in a containerized environment such as Kubernetes, each layer runs as a set of independent containers (Pods). This allows for independent scaling based on the specific load of each layer. For example, if the parsing layer has a high computational load, the number of container instances for the parsing layer service can be increased individually without modifying the receiving or forwarding layer.

[0209] A message queue is a middleware component used to pass messages in a distributed system. It employs a publish-subscribe or point-to-point pattern to achieve asynchronous communication. Message queues guarantee reliable message delivery, ordering, and decoupling.

[0210] Decoupled communication refers to the process where the receiving layer, parsing layer, and forwarding layer do not directly make network calls, but instead indirectly transmit data through message queues, reducing inter-layer dependencies and improving system elasticity and scalability.

[0211] In some embodiments, RabbitMQ can be used as the message queue. The receiving layer, acting as a producer, publishes the received streaming text data blocks as messages to a queue named `raw_text_queue`. The parsing layer, acting as a consumer, retrieves messages from this queue for processing. After processing, the parsing layer, acting as a producer again, publishes the text data blocks with placeholders to another queue named `parsed_text_queue` for the forwarding layer to consume and push to the client.

[0212] In some embodiments, Apache Kafka can be used as the message queue. The receiving layer sends raw text data blocks as records to the raw-data topic. The parsing layer, acting as a consumer group, subscribes to and processes the data on this topic, then sends the processing results to the processed-data topic. The forwarding layer subscribes to the processed-data topic, retrieves the data, and forwards it. This log-stream-based approach supports high throughput and data backtracking.

[0213] In some embodiments of this specification, physical and logical decoupling of the architecture is achieved by deploying each layer as an independent sub-device and using message queues for communication. This design allows each layer to scale independently based on load, improving resource utilization. Simultaneously, the buffering capacity of the message queues enhances system fault tolerance; even if a single sub-device fails, data loss or service interruption will not occur, significantly improving the system's scalability and robustness.

[0214] Some embodiments of this specification provide a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer executes a streaming data processing method based on an intermediate layer as described in any embodiment of this specification.

[0215] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0216] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0217] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0218] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0219] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0220] For each patent, patent application, patent application publication, and other material such as articles, books, specifications, publications, and documents referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0221] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A streaming data processing method based on an intermediate layer, characterized in that, The method is applied to an intermediate layer deployed between a server and a third-party large model, and the method includes: Receive streaming text data from the aforementioned third-party large model; Based on the streaming text data, resource reference tags are identified to determine the resource addresses corresponding to the resource reference tags; Generate a placeholder identifier corresponding to the resource address, and replace the resource reference tag with a placeholder tag, wherein the placeholder tag includes the placeholder identifier; The placeholder label and the streaming text data are forwarded to the client; wherein the client is configured to display the streaming text data in real time on the display page, and to locate and update the placeholder element corresponding to the image position on the display page according to the placeholder label.

2. The method according to claim 1, characterized in that, The method further includes: An asynchronous metadata query request is initiated for the resource address to obtain the local resource chapter information bound to the resource address based on the metadata mapping table; In response to the query of the local resource chapter information bound to the resource address, a first backfill event is generated and sent to the client, so that the client can replace the placeholder element with an image hyperlink including chapter jump based on the first backfill event; the first backfill event includes the placeholder identifier, the resource address and the local resource chapter information; In response to the failure to find the local resource chapter information bound to the resource address, a second backfill event is generated so that the client can replace the placeholder element with an image carrying a metadata-free tag based on the second backfill event; the second backfill event includes the metadata-free tag and is sent to the client.

3. The method according to claim 2, characterized in that, The asynchronous initiation of a metadata query request for the resource address, to obtain the local resource chapter information bound to the resource address according to the metadata mapping table, includes: The metadata query request is submitted to the asynchronous task queue, and the metadata query request includes the resource address and the placeholder identifier. A metadata query worker thread, independent of the text stream forwarding thread, executes a query task from the asynchronous task queue, including: using the resource address as the key, querying the metadata mapping table preloaded into the memory database to obtain the local resource chapter information, wherein the local resource chapter information includes the specification document name, chapter number, and page number information corresponding to the resource address.

4. The method according to claim 1, characterized in that, The step of identifying resource reference tags based on the streaming text data and determining the resource address corresponding to the resource reference tag includes: Establish a sliding window buffer corresponding to the client session, the sliding window buffer being used to cache the text characters of the streaming text data; The text characters in the sliding window buffer are matched using a preset regular expression to identify the resource reference tag; In response to the successful identification of the resource reference tag, the resource address is obtained by parsing from the tag structure.

5. The method according to claim 4, characterized in that, The step of matching text characters in the sliding window buffer using a preset regular expression to identify the resource reference tag includes: Record the index of the end position of the last matching operation; When new text characters are added to the sliding window buffer, regular expression matching based on the preset regular expression is performed on the sub-window interval that starts from the end position index and extends backward by a preset length. If the regular expression match is successful, the end position index is updated to the end position of this match.

6. The method according to claim 5, characterized in that, The method further includes: After the new text character is added to the sliding window buffer, it is determined whether the text character in the sliding window buffer exceeds the preset capacity limit; In response to the text characters in the sliding window buffer exceeding a preset capacity limit, based on the buffering time of the text characters, a portion of the text characters are removed from the sliding window buffer to release the capacity space of the sliding window buffer; The removed text characters are saved to the underlying reuse array corresponding to the sliding window buffer.

7. The method according to claim 1, characterized in that, The method further includes: After generating the placeholder identifier, a session record is created in the pre-generated placeholder registry. The session record is used to track the query status, resource address, and metadata of the placeholder identifier. Create a scheduled task for detecting the placeholder registry; In response to the detection in the placeholder registry that the placeholder identifier has not completed the metadata query within a preset time, the status of the placeholder identifier is marked as timed out, and a downgrade backfill event is sent to the client; In response to the detection of a placeholder identifier that has been backfilled or marked as timed out in the placeholder registry, the placeholder identifier that has been backfilled or marked as timed out is cleared from the placeholder registry after a preset delay time.

8. A streaming data processing system based on an intermediate layer, characterized in that, This includes an intermediate layer deployed between the server and a third-party large model, the intermediate layer comprising: The receiving module is configured to receive streaming text data from the third-party large model; The address determination module is configured to identify resource reference tags based on the streaming text data and determine the resource address corresponding to the resource reference tags. The identifier generation module is configured to generate a placeholder identifier corresponding to the resource address and replace the resource reference tag with a placeholder tag, wherein the placeholder tag includes the placeholder identifier; The forwarding module is configured to forward the placeholder tags and the streaming text data to the client; wherein the client is configured to display the streaming text data in real time on the display page, and to locate and update the placeholder element corresponding to the image position on the display page according to the placeholder tags.

9. An intermediate layer device for streaming data processing, characterized in that, The intermediate layer device is deployed between the server and the third-party large model, and the intermediate layer device includes: The receiving layer is configured to receive streaming text data from the third-party large model; The parsing layer is configured to identify resource reference tags based on the streaming text data, determine the resource address corresponding to the resource reference tag, generate a placeholder identifier corresponding to the resource address, and replace the resource reference tag with the placeholder tag, wherein the placeholder tag includes the placeholder identifier. A forwarding layer is configured to forward the placeholder tags and the streaming text data to a client; wherein the client is configured to display the streaming text data in real time on a display page, and to locate and update the placeholder element corresponding to the image position on the display page based on the placeholder tags.

10. The apparatus according to claim 9, characterized in that, The receiving layer, parsing layer, and forwarding layer are each independent sub-devices, and they communicate with each other through message queues.

Images