Method for developing AI application based on responsive streaming interaction
By adopting a layered, responsive, and fluid interactive architecture, the problems of high interaction latency and low data processing efficiency in existing technologies are solved, enabling low-latency interaction and efficient data processing in AI applications, thereby improving user experience and system performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- IOL WUHAN INFORMATION TECH CO LTD
- Filing Date
- 2025-12-23
- Publication Date
- 2026-05-12
AI Technical Summary
In existing technologies, AI application development suffers from significant interaction delays when processing large-scale data or complex model inference. The data processing links are tightly coupled, and there is a lack of a unified streaming event protocol, resulting in poor user experience and limited system throughput. It is also difficult to adapt to dynamic switching of multiple models and integration of tool calls.
The architecture employs a layered, responsive streaming design, comprising a streaming data pipeline layer, an asynchronous event processing layer, a model inference layer, and a user interface response layer. Through streaming data processing, asynchronous event processing, and responsive interface updates, it enables real-time data flow and real-time interface updates, supporting multimodal data chunked transmission and incremental inference.
It achieves low-latency interaction and efficient data processing, improves user experience and system performance, reduces interaction response latency, supports throughput and resource utilization in high-concurrency scenarios, and enhances the system's robustness and data consistency in abnormal scenarios.
Smart Images
Figure CN122018953A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for developing AI applications, specifically a method for developing AI applications based on responsive streaming interaction, belonging to the field of artificial intelligence application development technology. Background Technology
[0002] In the field of artificial intelligence application development, traditional interaction models mostly adopt a request-response model, where after the user inputs complete data, the system needs to perform full data processing and generate the final result before returning it to the user. This model leads to significant interaction delays when handling large-scale data or complex model inference.
[0003] In the prior art, 1) the interactive response method and related devices, electronic devices and storage media disclosed in CN116954461A use an artificial intelligence model to determine whether the feedback data can respond to the user's request, and introduce a network query mechanism when it cannot respond. Although this improves the accuracy of the response, its core still relies on the complete request-response cycle and fails to solve the problem of real-time block transmission and incremental inference of data streams. Especially when processing multimodal data or long text dialogues, users still need to wait for the full processing to be completed, resulting in a poor experience; 2) the streaming dialogue interface based on a large model disclosed in CN118351831A supports the streaming transmission of voice and text, but its architecture still focuses on the unidirectional flow of data and backend batch processing. It lacks the layered decoupling of data streams and the event-driven asynchronous scheduling mechanism, which leads to limited system throughput and delayed interface updates in high-concurrency scenarios.
[0004] In summary, the common shortcomings of existing technologies are as follows: First, the data processing chain is tightly coupled, with synchronous blockages at each stage from input to inference to interface updates, making it impossible to achieve truly low-latency pipeline operations; second, there is a lack of a unified streaming event protocol to coordinate incremental data transmission, model inference, and interface state synchronization, resulting in less smooth interactive responses; and third, the architecture lacks scalability, making it difficult to adapt to the needs of complex AI application scenarios such as dynamic switching of multiple models and integration of tool calls.
[0005] Therefore, there is an urgent need for a development methodology that can deeply integrate streaming data pipelines, asynchronous event processing, model streaming inference, and responsive interface updates to systematically solve problems such as high interaction latency, low data processing efficiency, and fragmented user experience. Summary of the Invention
[0006] The purpose of this invention is to provide a method for developing AI applications based on responsive streaming interaction in order to solve at least one of the above-mentioned technical problems.
[0007] The present invention achieves the above objectives through the following technical solution: a method for developing AI applications based on responsive streaming interaction, including a layered responsive streaming interaction architecture, which includes a streaming data pipeline layer, an asynchronous event processing layer, a model inference layer, and an interface response layer;
[0008] The method includes the following steps:
[0009] Construct a streaming data processing link, and perform incremental transmission and decoupling processing of input data through a streaming data pipeline layer;
[0010] Non-blocking task scheduling is achieved through an asynchronous event processing layer based on an event-driven architecture;
[0011] Through the model inference layer, incremental inference is performed on the received streaming data, and corresponding model output events are triggered.
[0012] Through the UI response layer, we subscribe to state change events and drive the user interface to perform real-time responsive updates based on state snapshots and incremental update information.
[0013] As a further aspect of the present invention: the streaming data pipeline layer includes a responsive streaming library, a backpressure control mechanism, and multimodal data partitioning, specifically including:
[0014] The streaming data pipeline layer uses a responsive streaming library to build data streams and supports a backpressure control mechanism. Input data includes one or more multimodal data such as text, images, or audio, and is streamed in a chunked manner.
[0015] As a further aspect of the present invention: a unified event model is defined in the asynchronous event processing layer, which includes event type, timestamp, and payload attributes.
[0016] As a further aspect of the present invention: the event types include:
[0017] Lifecycle events are used to monitor the running status of AI agents;
[0018] Data stream events are used to deliver incremental data or indicate the end of a stream;
[0019] Tool call events are used to manage calls to external APIs and the return of results.
[0020] As a further aspect of the present invention: the model inference layer includes a streaming inference adapter, a dynamic batch processing engine, and attention mechanism optimization. The model inference layer integrates an AI model and is compatible with different model frameworks through the streaming inference adapter. The model inference layer supports streaming inference. During the inference process, a model output event is triggered once for each output unit generated, and the inference process can be terminated in advance by terminating the termination event.
[0021] As a further aspect of the present invention: the interface response layer includes a responsive UI framework, a state snapshot-incremental approach, and automatic state subscription. The interface response layer is implemented based on the responsive UI framework and uses a snapshot-incremental approach for state management, specifically including:
[0022] Pass a complete state snapshot during initial load;
[0023] Only incremental update information of the changed fields is transmitted when the state changes;
[0024] The UI response layer achieves automatic updates by subscribing to state streams.
[0025] As a further aspect of the present invention, the method also includes a fault tolerance and recovery mechanism, specifically including:
[0026] The breakpoint resume mechanism records the identifier of the last processed data block and resumes transmission from the breakpoint after the network is restored.
[0027] The status verification mechanism involves the front-end periodically sending status verification events, comparing the hash values with the back-end status, and triggering a full synchronization when there is a discrepancy.
[0028] As a further aspect of the present invention: the method employs an event flow communication protocol, which includes, but is not limited to, defining standard event patterns and event flow patterns.
[0029] The beneficial effects of this invention are:
[0030] 1) This invention utilizes a responsive streaming interaction architecture, encompassing core technology modules such as streaming data pipelines, asynchronous event processing, and responsive programming models. By constructing an AI application development framework based on streaming data processing, it enables real-time data flow, streaming inference of AI models, and responsive updates of the user interface, establishing an end-to-end streaming processing link from data input to result output. This achieves low-latency interaction and efficient data processing in AI applications, improving user experience and system performance.
[0031] 2) This invention utilizes a streaming data pipeline layer and backpressure control mechanism to achieve chunked data transmission and multimodal streaming processing, effectively reducing interactive response latency. Users can obtain incremental results without waiting for all data to complete, significantly improving real-time performance. Through an asynchronous event processing layer and a unified event model, data flow, tool calls, and lifecycle events are decoupled, supporting non-blocking task scheduling and improving system throughput and resource utilization in high-concurrency scenarios. The model inference layer, through a streaming inference adapter and incremental output mechanism, enables AI models to generate results while receiving data, and combined with dynamic termination functionality, further reduces unnecessary computation and improves inference efficiency. The interface response layer, based on a state snapshot-incremental mode and a responsive UI framework, achieves real-time subscription and updating of interface states, ensuring the synchronization of user operations and visual feedback, and eliminating the experience fragmentation caused by interface update lag. Through fault tolerance and recovery mechanisms such as breakpoint resume and state verification, the robustness and data consistency of the system in abnormal scenarios are enhanced. Attached Figure Description
[0032] Figure 1 This is a schematic diagram of the process framework of the present invention;
[0033] Figure 2 This is a schematic diagram of the streaming data pipeline layer structure of the present invention;
[0034] Figure 3 This is a schematic diagram of the asynchronous event processing layer structure of the present invention;
[0035] Figure 4 This is a schematic diagram of the model inference layer structure of the present invention;
[0036] Figure 5 This is a schematic diagram of the interface response layer structure of the present invention;
[0037] Figure 6 This is a flowchart illustrating Embodiment Six of the present invention. Detailed Implementation
[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0039] Example 1, as Figure 1 As shown, this embodiment provides a method for developing AI applications based on responsive streaming interaction, including a layered responsive streaming interaction architecture, which includes a streaming data pipeline layer, an asynchronous event processing layer, a model inference layer, and an interface response layer.
[0040] The method includes the following steps:
[0041] Construct a streaming data processing link, and perform incremental transmission and decoupling processing of input data through a streaming data pipeline layer;
[0042] Non-blocking task scheduling is achieved through an asynchronous event processing layer based on an event-driven architecture;
[0043] Through the model inference layer, incremental inference is performed on the received streaming data, and corresponding model output events are triggered.
[0044] Through the UI response layer, we subscribe to state change events and drive the user interface to perform real-time responsive updates based on state snapshots and incremental update information.
[0045] Example 2, in addition to all the technical features included in Example 1, also includes: Figure 2 As shown, the streaming data pipeline layer includes a responsive streaming library, a backpressure control mechanism, and multimodal data partitioning, specifically including:
[0046] The streaming data pipeline layer uses a responsive streaming library to build data streams and supports a backpressure control mechanism. Input data includes one or more multimodal data such as text, images, or audio, and is streamed in a chunked manner.
[0047] The system ensures the stability of data streams and system reliability under high concurrency by using a responsive streaming library and backpressure mechanism, preventing resource overload, while also supporting multimodal data chunked transmission.
[0048] like Figure 3 As shown, the asynchronous event processing layer defines a unified event model, which includes event type, timestamp, and payload attributes. By defining a unified event format, a standardized and structured information exchange basis is provided for communication between modules within the system, reducing the coupling between system modules and improving the clarity and maintainability of the architecture.
[0049] Event types include:
[0050] Lifecycle events are used to monitor the running status of AI agents;
[0051] Data stream events are used to deliver incremental data or indicate the end of a stream;
[0052] Tool call events are used to manage calls to external APIs and the return of results.
[0053] like Figure 4As shown, the model inference layer includes a streaming inference adapter, a dynamic batch processing engine, and attention mechanism optimization. The model inference layer integrates an AI model and is compatible with different model frameworks through the streaming inference adapter. The model inference layer supports streaming inference, triggering a model output event once for each output unit generated during the inference process, and supports early termination of the inference process through a termination event.
[0054] like Figure 5 As shown, the interface response layer includes a responsive UI framework, state snapshot-incremental, and automatic state subscription. The interface response layer is implemented based on the responsive UI framework and uses a snapshot-incremental approach for state management, specifically including:
[0055] Pass a complete state snapshot during initial load;
[0056] Only incremental update information of the changed fields is transmitted when the state changes;
[0057] The UI response layer achieves automatic updates by subscribing to state streams.
[0058] By adopting a snapshot-incremental state management model, only changes need to be transmitted after the initial load, which greatly reduces the amount of network data transmission and bandwidth consumption. Combined with the automatic subscription update mechanism of the responsive UI framework, it ensures that the update of the user interface is highly synchronized with the changes in the backend state, and achieves smooth and real-time interface feedback.
[0059] Example 3: In addition to all the technical features included in Example 1, this example also includes a fault tolerance and recovery mechanism, specifically including:
[0060] The breakpoint resume mechanism records the identifier of the last processed data block and resumes transmission from the breakpoint after the network is restored.
[0061] The status verification mechanism involves the front-end periodically sending status verification events, comparing the hash values with the back-end status, and triggering a full synchronization when there is a discrepancy.
[0062] The breakpoint resume mechanism improves the robustness of the system in non-ideal network environments or abnormal situations. Breakpoint resume avoids repeated data transmission and processing, thus improving efficiency. The status verification mechanism ensures the consistency of the front-end and back-end status, preventing functional errors or display abnormalities caused by status asynchrony, thereby enhancing the stability and reliability of the system.
[0063] This method employs an event stream communication protocol, which includes, but is not limited to, defining standard event patterns and event stream patterns.
[0064] Event-driven communication protocols specifically include:
[0065] Define standard event format:
[0066] {
[0067] "type":"TextMessageStart",
[0068] "timestamp":1725097600000,
[0069] "messageId":"msg_123",
[0070] "role":"assistant",
[0071] "content": "Streaming content is delivered via subsequent events"
[0072] }
[0073] Example 4, as Figure 6 As shown, this embodiment takes real-time dialogue as an example, and the communication process is as follows:
[0074] User input: The front end sends an input event (TextInput event) containing chunks of text data.
[0075] Streaming:
[0076] The data pipeline layer receives input events and transmits them in chunks to the model inference layer;
[0077] Each time the model inference layer generates an output unit (Token), it triggers a ModelOutput event, carrying incremental text.
[0078] Interface update:
[0079] The front-end subscribes to the ModelOutput event to append text to the dialog box in real time. At the same time, if the model calls an external tool (such as a search API), the tool call event (ToolInvoke event) is triggered, and the front-end updates the interface state accordingly.
[0080] Result returned: After the model inference is completed, the StreamEnd event is triggered, and the frontend terminates the corresponding loading state.
[0081] This embodiment clearly demonstrates how the present invention can be implemented in key application scenarios. Through event-driven and streaming processing, it achieves seamless connection and real-time linkage between user input, model inference, tool invocation, and interface feedback, specifically solving the problems of feedback delay and fragmented experience in traditional dialogue applications.
[0082] Working process and principle: The entire processing is decomposed into four collaborative layers: the streaming data pipeline layer, the asynchronous event processing layer, the model inference layer, and the interface response layer. When a user initiates an interaction and inputs data, it first enters the streaming data pipeline layer. This layer uses a reactive streaming library to build the data stream and divides the data into small chunks for incremental transmission. At the same time, a backpressure mechanism intelligently controls the data flow rate to prevent the system from overloading under high pressure.
[0083] The processed data blocks are encapsulated into standardized events, which are scheduled by the asynchronous event processing layer. These events follow a unified event model, including attributes such as type, timestamp, and payload. Specifically, they are divided into: lifecycle events for monitoring the running status of AI agents, data stream events for transmitting incremental data, and tool call events for managing external API calls. The event-driven architecture ensures non-blocking and efficient asynchronous execution of tasks.
[0084] The event is passed to the model inference layer, which integrates various AI models through the adapter pattern and starts the streaming inference process model. It does not need to wait for all data input to be completed, but performs inference while receiving data. Each time an output unit such as a token is generated, a model output event is immediately triggered, thereby achieving incremental and real-time output of results. It also supports the early termination of inference through termination events to respond to user interruption operations.
[0085] The incremental results output by the final model are subscribed to as state update events by the UI response layer. This layer uses a snapshot-incremental pattern for state management based on a responsive UI framework. That is, a complete state snapshot is obtained during initial loading, and only incremental update information that has changed is received and applied thereafter. UI components achieve millisecond-level responsive updates by automatically subscribing to these state streams, so that users can see the word-by-word output of the dialogue content or the dynamic changes of the recommendation results in real time.
[0086] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
[0087] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A method for developing AI applications based on responsive streaming interaction, comprising a layered responsive streaming interaction architecture, characterized in that: The responsive streaming interaction architecture includes a streaming data pipeline layer, an asynchronous event processing layer, a model inference layer, and a user interface response layer. The method includes the following steps: Construct a streaming data processing link, and perform incremental transmission and decoupling processing of input data through a streaming data pipeline layer; Non-blocking task scheduling is achieved through an asynchronous event processing layer based on an event-driven architecture; Through the model inference layer, incremental inference is performed on the received streaming data, and corresponding model output events are triggered. Through the UI response layer, we subscribe to state change events and drive the user interface to perform real-time responsive updates based on state snapshots and incremental update information.
2. The method according to claim 1, characterized in that: The streaming data pipeline layer includes a responsive streaming library, a backpressure control mechanism, and multimodal data partitioning, specifically including: The streaming data pipeline layer uses a responsive streaming library to construct data streams and supports a backpressure control mechanism; the input data includes one or more multimodal data such as text, images or audio, and is streamed in a block-based manner through multimodal data segmentation.
3. The method according to claim 1, characterized in that: The asynchronous event processing layer defines a unified event model, which includes event type, timestamp, and payload attributes.
4. The method according to claim 3, characterized in that: The event types include: Lifecycle events are used to monitor the running status of AI agents; Data stream events are used to deliver incremental data or indicate the end of a stream; Tool call events are used to manage calls to external APIs and the return of results.
5. The method according to claim 1, characterized in that: The model inference layer includes a streaming inference adapter, a dynamic batch processing engine, and attention mechanism optimization. The model inference layer integrates an AI model and is compatible with different model frameworks through the streaming inference adapter. The model inference layer supports streaming inference, triggering a model output event for each output unit generated during the inference process, and supports early termination of the inference process through a termination event.
6. The method according to claim 1, characterized in that: The interface response layer includes a responsive UI framework, state snapshot-incremental, and automatic state subscription. The interface response layer is implemented based on the responsive UI framework and uses a snapshot-incremental approach for state management, specifically including: Pass a complete state snapshot during initial load; Only incremental update information of the changed fields is transmitted when the state changes; The UI response layer achieves automatic updates by subscribing to state streams.
7. The method according to claim 1, characterized in that, The method also includes fault tolerance and recovery mechanisms, specifically including: The breakpoint resume mechanism records the identifier of the last processed data block and resumes transmission from the breakpoint after the network is restored. The status verification mechanism involves the front-end periodically sending status verification events, comparing the hash values with the back-end status, and triggering a full synchronization when they are inconsistent.
8. The method according to claim 1, characterized in that: The method employs an event flow communication protocol, which includes, but is not limited to, defining standard event patterns and event flow patterns.