A large model asynchronous invocation and streaming response bridging method in enterprise instant messaging machine

Abstract

The application relates to the technical field of enterprise instant communication robots, and discloses a large model asynchronous calling and streaming response bridging method in an enterprise instant communication robot, which comprises the following steps: establishing a global singleton WebSocket connection service and registering a platform event processor; receiving and analyzing user messages through a Webhook interface; initiating asynchronous storage of the messages; creating a placeholder card and initiating sending, and then quickly returning a receiving confirmation response; obtaining card positioning information in the background; generating a bridging association identifier by a streaming response bridging processor; establishing a bridging relationship between the bridging association identifier and the card positioning information and subscribing to a large model streaming response event; starting an intelligent agent executor to call the large model; receiving incremental response content; splicing the incremental response content according to an event receiving sequence or an event serial number sequence to form accumulated response content; and continuously writing the accumulated response content into the same placeholder card through a card updating interface, so that the large model response content can be stably and continuously displayed in an enterprise instant communication session.

Description

Technical Field

[0001] This invention relates to the field of enterprise instant messaging robot technology, and in particular to a method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot. Background Technology

[0002] With the widespread application of enterprise instant messaging platforms in scenarios such as office collaboration, enterprise knowledge Q&A, business approval, intelligent customer service, and internal information retrieval, more and more enterprises are connecting intelligent agents and large models to enterprise instant messaging platforms through robot interfaces. After users send questions to the robot in the chat window of the enterprise instant messaging platform, the robot calls the large model to generate answers and returns the answers to the corresponding instant messaging chat.

[0003] Existing large models typically require processes such as reading the session context, constructing prompts, model inference, and gradually generating text when generating response content. Their response time is significantly longer than that of ordinary message interfaces. When enterprise instant messaging platforms push message events to robots, they usually require the Webhook interface to return a confirmation response within a specified time. If the Webhook interface synchronously waits for the large model to generate complete response content before returning, it can easily cause the Webhook processing thread to be occupied for a long time, the enterprise instant messaging platform to repeatedly push message events, the robot to repeatedly create large model generation tasks, or message processing to time out.

[0004] To address the aforementioned issues, a search revealed a patent with publication number CN110945840B that discloses a method and system for providing an embedded application associated with a messaging application. This system can display interactive cards within a chat interface and continuously update the information in the interactive cards based on the embedded application status, user operations, and related data. While this solution can update the content of interactive cards in messaging applications, it does not address scenarios where large model generation takes a long time and incremental content is returned in stages via streaming events. Instead, it discloses a collaborative processing mechanism between Webhook for rapid response, asynchronous sending of placeholder cards, acquisition of card location information, pre-establishment of bridging relationships, and continuous feedback of accumulated response content.

[0005] In the actual development of large-scale robot models, if the large-scale model generation task is started before the placeholder cards have been sent and the card positioning information has been obtained, the large-scale model may have already generated the first batch of incremental response content. At this time, the streaming response event cannot locate the corresponding placeholder card, which can easily cause the first segment of content to be lost.

[0006] If the current incremental text is only written to the placeholder card each time, without being concatenated to the cumulative response content, the card update interface using the overall overwrite method may cause previously displayed content to be overwritten by subsequent incremental text. At the same time, when multiple message processing objects establish WebSocket connections or repeatedly register event handlers with the event dispatcher, problems such as inconsistent connection states, repeated consumption of the same streaming response event, and repeated writing of large model response content to the card may occur. In addition, if the streaming event subscription is not unsubscribed, the bridging mapping is not deleted, and the cumulative response content cache is not released after the large model generation task is completed, the robot may experience a continuous accumulation of mapping data and cached data during long-term operation.

[0007] Therefore, a method for bridging large-scale asynchronous calls and streaming responses in enterprise instant messaging robots is needed. This method should ensure that the Webhook interface returns a confirmation response quickly while reasonably coordinating the sending of placeholder cards, acquisition of card location information, establishment of bridging relationships, initiation of large-scale task, accumulation of streaming responses, continuous card updates, and resource cleanup after the task ends. Summary of the Invention

[0008] The purpose of this invention is to provide a method for bridging asynchronous calls and streaming responses of large models in enterprise instant messaging robots. This addresses the problem that existing large models typically require processes such as reading the session context, constructing prompts, model inference, and gradually generating text when generating response content, resulting in significantly longer response times than ordinary message interfaces. When enterprise instant messaging platforms push message events to robots, they usually require the Webhook interface to return a confirmation response within a specified time. If the Webhook interface synchronously waits for the large model to generate complete response content before returning, it can easily lead to problems such as the Webhook processing thread being occupied for a long time, the enterprise instant messaging platform repeatedly pushing message events, the robot repeatedly creating large model generation tasks, or message processing timeouts.

[0009] In this technical solution, a method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot includes the following steps:

[0010] Step S1: The connection service is started, a global singleton connection service for the enterprise instant messaging platform is created, a WebSocket long connection with the enterprise instant messaging platform is established through the global singleton connection service, and the enterprise instant messaging platform event handler is registered with the event dispatcher to maintain the continuous connection state and listen for the connection status event and asynchronous platform event of the enterprise instant messaging platform.

[0011] Step S2, message reception and parsing: Receive user messages sent by the enterprise instant messaging platform through the Webhook interface, parse the user messages to obtain session information, user information and message content, and submit the parsed user messages to the message pipeline processor.

[0012] Step S3, asynchronous message storage: The message pipeline processor initiates the operation of asynchronously writing the parsed user message into the message storage unit, so that the message storage process does not block the response process of the Webhook interface.

[0013] Step S4, placeholder card processing: The message pipeline processor creates a placeholder card for displaying the large model response content based on the session information, and initiates the operation of sending the placeholder card to the enterprise instant messaging session where the user message is located. After the placeholder card is sent, the card location information for locating the placeholder card is returned.

[0014] Step S5, Webhook responds quickly. After initiating the placeholder card sending operation, it returns a Webhook reception confirmation response to the enterprise instant messaging platform, causing the Webhook interface to end this synchronization process without waiting for the placeholder card sending operation to complete or the large model to return the complete response content.

[0015] Step S6, asynchronous process continuation: In the asynchronous processing process after the Webhook synchronous processing is completed, wait for the placeholder card sending operation to return the card positioning information, combine the message content with the preset prompt word template to form the large model input prompt word, and pass the large model input prompt word to the intelligent agent executor.

[0016] Step S7, pre-establishment of bridging relationship: Before starting the large model generation task, the streaming response bridging processor generates a bridging association identifier that can uniquely identify the current generation process for this large model generation process, establishes the bridging relationship between the bridging association identifier and the card positioning information, and subscribes to the large model streaming response event output by the intelligent agent executor through the event dispatcher.

[0017] Step S8: The large model is executed asynchronously. After the bridging relationship is established and the large model streaming response event subscription is completed, the agent executor is started to call the large model to execute the generation task, and the bridging association identifier is passed to the agent executor as a task call parameter or streaming callback context identifier. When the agent executor returns the large model generation task identifier, the correspondence between the large model generation task identifier and the bridging association identifier is established. After the generation task is started, the large model does not wait for the large model to return the complete response content. The global singleton connection service continuously maintains the WebSocket long connection, and the event dispatcher keeps the event listening state.

[0018] Step S9, Streaming Response Bridging: When a large model streaming response event is received, the incremental response content and at least one of the following are obtained from the large model streaming response event: bridging association identifier, large model generation task identifier, or streaming callback context identifier. Based on the obtained information, the corresponding card positioning information is queried. The incremental response content is then concatenated to the cumulative response content of the corresponding large model generation process according to the receiving order of the streaming response event or the event sequence number order. The updated cumulative response content is then converted into card update data supported by the enterprise instant messaging platform.

[0019] Step S10: Card streaming backfeed. A card update request is generated based on the card location information and card update data. The updated cumulative response content is written to the same placeholder card through the card update interface provided by the enterprise instant messaging platform. The response content generated by the large model is gradually displayed in the enterprise instant messaging session along with the streaming response event until the large model generation task is completed.

[0020] Preferably, in step S1, the global singleton connection service is a singleton WebSocket connection service. The singleton WebSocket connection service manages the WebSocket connection instance, event dispatcher, and connection startup status in a unified manner, so that the enterprise instant messaging robot can share the same connection service instance during operation.

[0021] Preferably, the singleton WebSocket connection service is equipped with a startup flag. Before starting the singleton WebSocket connection service, the startup flag is read. When the startup flag indicates that the singleton WebSocket connection service has not yet started, a WebSocket long connection is established, an enterprise instant messaging platform event handler is registered, and the startup flag is updated. When the startup flag indicates that the singleton WebSocket connection service has already started, the repeated establishment of WebSocket long connections and the repeated registration of enterprise instant messaging platform event handlers are stopped.

[0022] Preferably, the singleton WebSocket connection service encapsulates the WebSocket connection instance, event dispatcher, and start flag through private fields, and controls the WebSocket connection instance through the externally provided start interface and message sending interface, so as to avoid connection state confusion caused by multiple message pipeline processors establishing WebSocket long connections separately.

[0023] Preferably, in steps S2 to S8, the message pipeline processor sequentially executes message parsing, initiates asynchronous message storage, creates placeholder cards, initiates placeholder card sending, and returns a Webhook reception confirmation response. In the asynchronous processing flow after the Webhook synchronous processing ends, the message pipeline processor obtains the card location information returned by the placeholder card sending operation and performs prompt word injection. The streaming response bridging processor generates a bridging association identifier for this large model generation process, establishes a bridging relationship between the bridging association identifier and the card location information, and completes the large model streaming response event subscription through the event dispatcher. After completing the bridging relationship establishment and event subscription, the message pipeline processor starts the intelligent agent executor to execute the large model generation task.

[0024] Preferably, in step S3, after initiating the asynchronous message storage operation, the message pipeline processor continues to execute the placeholder card creation and placeholder card sending operations without waiting for the asynchronous message storage operation to complete, so that the message storage process and the placeholder card processing process are executed asynchronously.

[0025] Preferably, in steps S4 to S8, the placeholder card includes at least a response content display area. The response content display area is written with waiting response information when the placeholder card is created. After initiating the placeholder card sending operation, the message pipeline processor generates and returns a Webhook confirmation response. After the Webhook synchronization process is completed, it waits for the placeholder card sending operation to finish and obtains the card location information. After obtaining the card location information, the message pipeline processor sends the card location information to the streaming response bridge processor. After the streaming response bridge processor establishes a bridging relationship between the bridge association identifier and the card location information and completes the subscription to the large model streaming response event, the message pipeline processor then starts the agent executor to execute the large model generation task, to prevent the streaming response event generated by the large model generation task from failing to write to the corresponding placeholder card due to the lack of card location information.

[0026] Preferably, in steps S6 to S8, the message pipeline processor injects the user message content into the preset prompt word template and passes the resulting large model input prompt word to the intelligent agent executor. After the intelligent agent executor starts the large model generation task, it encapsulates the incremental response content generated during the large model generation process into a large model streaming response event and outputs it through a callback method, instead of using the complete response content generated by the large model as the synchronous return value of the Webhook interface.

[0027] Preferably, in steps S7 and S8, the bridging association identifier is generated before the large model generation task starts, and is used to uniquely correspond to this large model generation process. When the intelligent agent executor starts the large model generation task, it receives the bridging association identifier and writes the bridging association identifier into the task call parameters or the streaming callback context identifier. When the intelligent agent executor returns the large model generation task identifier, it establishes a correspondence between the large model generation task identifier and the bridging association identifier. The card positioning information includes at least one of the placeholder card identifier, instant messaging message identifier, or card update address. After receiving the large model streaming response event, the event dispatcher queries the corresponding card positioning information according to at least one of the bridging association identifier, the large model generation task identifier, or the streaming callback context identifier, and transfers the large model streaming response event to the streaming response bridging processor associated with the corresponding placeholder card.

[0028] Preferably, in steps S9 and S10, the incremental response content is the text content output in stages during the generation of the large model. Each time a large model streaming response event is received, the streaming response bridging processor concatenates the incremental response content to the cumulative response content of the corresponding large model generation process according to the receiving order of the streaming response events or the event sequence number carried by the streaming response events. It generates a card update request based on the card positioning information and the updated cumulative response content, and writes the cumulative response content into the same placeholder card through the card PUT update method. After the large model generation task is completed, the final cumulative response content is written into the placeholder card, the subscription to the large model streaming response event corresponding to this large model generation process is released, the correspondence between the bridging association identifier, the large model generation task identifier and the card positioning information is deleted, and the corresponding cumulative response content cache is released.

[0029] Compared with existing technologies, the asynchronous call and streaming response bridging method for large models in an enterprise instant messaging robot of the present invention has the following advantages:

[0030] 1. By quickly returning a Webhook confirmation response after initiating asynchronous message storage and placeholder card sending operations, the large model call process no longer occupies the Webhook synchronous processing thread. This reduces the probability of the enterprise instant messaging platform repeatedly pushing message events due to response timeout. At the same time, a global singleton WebSocket connection service is used to uniformly manage connection instances, event dispatchers, and startup states. The startup flag prevents duplicate WebSocket connection establishment and duplicate event handler registration, thus avoiding connection state chaos caused by multiple connection instances and duplicate consumption of the same streaming response event.

[0031] 2. After obtaining the card location information in the background, the streaming response bridging processor first generates a bridging association identifier, establishes a bridging relationship between the bridging association identifier and the card location information, and completes the streaming event subscription. Then, the message pipeline processor starts the large model generation task, ensuring that the corresponding card location path is available before the large model starts outputting incremental content. This avoids the problem that the first batch of streaming response events cannot locate the card when the placeholder card has not been sent. By writing the bridging association identifier into the task call parameters or streaming callback context identifier, and establishing the correspondence between the task identifier and the bridging association identifier after the large model returns the task identifier, it can adapt to the actual call process where the large model task identifier can only be generated after the task starts.

[0032] 3. Upon receiving a streaming response event, the incremental response content is appended to the cumulative response content, and the updated cumulative response content is used to generate card update data, instead of simply writing the current incremental fragment to the placeholder card. This avoids the overall overwrite card update interface from overwriting the previously displayed content. Even if the card update interface call fails, the cumulative response content is still retained, and subsequent incremental response content can continue to be appended, avoiding the loss of response content due to a single update failure.

[0033] 4. After the large model generation task is completed, unsubscribe from streaming events, delete bridging mappings, and release the cached cumulative response content. This enables the generation process of each model to form a complete lifecycle of creation, execution, backfeeding, and cleanup, reducing the risk of continuous accumulation of bridging relationships and cumulative caches during long-term operation of enterprise instant messaging robots. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall system architecture of the present invention;

[0035] Figure 2 This is a flowchart of the asynchronous call and streaming response bridging method for large models in this invention;

[0036] Figure 3 This is a schematic diagram illustrating the detailed process of bridging correlation and flow response recharge in this invention. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0038] Please see Figure 1-3 A method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot includes the following steps:

[0039] Step S1: The connection service is started, a global singleton connection service for the enterprise instant messaging platform is created, a WebSocket long connection with the enterprise instant messaging platform is established through the global singleton connection service, and the enterprise instant messaging platform event handler is registered with the event dispatcher to maintain the continuous connection state and listen for the connection status event and asynchronous platform event of the enterprise instant messaging platform.

[0040] Step S2, message reception and parsing: Receive user messages sent by the enterprise instant messaging platform through the Webhook interface, parse the user messages to obtain session information, user information and message content, and submit the parsed user messages to the message pipeline processor.

[0041] Step S3, asynchronous message storage: The message pipeline processor initiates the operation of asynchronously writing the parsed user message into the message storage unit, so that the message storage process does not block the response process of the Webhook interface.

[0042] Step S4, placeholder card processing: The message pipeline processor creates a placeholder card for displaying the large model response content based on the session information, and initiates the operation of sending the placeholder card to the enterprise instant messaging session where the user message is located. After the placeholder card is sent, the card location information for locating the placeholder card is returned.

[0043] Step S5, Webhook responds quickly. After initiating the placeholder card sending operation, it returns a Webhook reception confirmation response to the enterprise instant messaging platform, causing the Webhook interface to end this synchronization process without waiting for the placeholder card sending operation to complete or the large model to return the complete response content.

[0044] Step S6, asynchronous process continuation: In the asynchronous processing process after the Webhook synchronous processing is completed, wait for the placeholder card sending operation to return the card positioning information, combine the message content with the preset prompt word template to form the large model input prompt word, and pass the large model input prompt word to the intelligent agent executor.

[0045] Step S7, pre-establishment of bridging relationship: Before starting the large model generation task, the streaming response bridging processor generates a bridging association identifier that can uniquely identify the current generation process for this large model generation process, establishes the bridging relationship between the bridging association identifier and the card positioning information, and subscribes to the large model streaming response event output by the intelligent agent executor through the event dispatcher.

[0046] Step S8: The large model is executed asynchronously. After the bridging relationship is established and the large model streaming response event subscription is completed, the agent executor is started to call the large model to execute the generation task, and the bridging association identifier is passed to the agent executor as a task call parameter or streaming callback context identifier. When the agent executor returns the large model generation task identifier, the correspondence between the large model generation task identifier and the bridging association identifier is established. After the generation task is started, the large model does not wait for the large model to return the complete response content. The global singleton connection service continuously maintains the WebSocket long connection, and the event dispatcher keeps the event listening state.

[0047] Step S9, Streaming Response Bridging: When a large model streaming response event is received, the incremental response content and at least one of the following are obtained from the large model streaming response event: bridging association identifier, large model generation task identifier, or streaming callback context identifier. Based on the obtained information, the corresponding card positioning information is queried. The incremental response content is then concatenated to the cumulative response content of the corresponding large model generation process according to the receiving order of the streaming response event or the event sequence number order. The updated cumulative response content is then converted into card update data supported by the enterprise instant messaging platform.

[0048] Step S10: Card streaming backfeed. A card update request is generated based on the card location information and card update data. The updated cumulative response content is written to the same placeholder card through the card update interface provided by the enterprise instant messaging platform. The response content generated by the large model is gradually displayed in the enterprise instant messaging session along with the streaming response event until the large model generation task is completed.

[0049] Furthermore, in step S1, the global singleton connection service is a singleton WebSocket connection service. The singleton WebSocket connection service manages the WebSocket connection instance, event dispatcher, and connection startup status in a unified manner, enabling enterprise instant messaging robots to share the same connection service instance during operation. By setting up the global singleton connection service, WebSocket connection establishment, connection status maintenance, platform event listening, and event dispatch can be completed in the same service instance. This avoids the problems of repeated occupation of connection resources, repeated registration of event handlers, and multiple consumption of the same platform event caused by multiple message processing processes creating WebSocket connections separately. At the same time, it enables asynchronous large model tasks corresponding to different user messages to share a continuously running event receiving channel, providing a unified connection foundation for the stable reception and accurate forwarding of subsequent streaming response events.

[0050] Furthermore, the singleton WebSocket connection service is equipped with a startup flag. Before starting the singleton WebSocket connection service, the startup flag is read. When the startup flag indicates that the singleton WebSocket connection service has not yet started, a WebSocket long connection is established, an enterprise instant messaging platform event handler is registered, and the startup flag is updated. When the startup flag indicates that the singleton WebSocket connection service has started, the repeated establishment of WebSocket long connections and repeated registration of enterprise instant messaging platform event handlers are stopped. By setting the startup flag, the startup status of the connection service can be uniformly judged when the robot is initialized, the service is reloaded, or multiple message requests trigger the connection startup operation concurrently. This ensures that the WebSocket long connection and the enterprise instant messaging platform event handler are only validly initialized once, avoiding duplicate connections that could lead to abnormal platform connection numbers, duplicate event callbacks, and duplicate writing of streaming response content to placeholder cards, thereby improving the idempotency and operational stability of the connection service startup process.

[0051] Furthermore, the singleton WebSocket connection service encapsulates the WebSocket connection instance, event dispatcher, and startup flag through private fields, and controls the WebSocket connection instance through the externally provided startup and message sending interfaces. This avoids connection state confusion caused by multiple message pipeline processors establishing separate WebSocket long connections. By encapsulating the connection instance, event dispatcher, and startup state through private fields, external message pipeline processors cannot directly modify the WebSocket connection state, but can only call the connection service through the unified startup and message sending interfaces. This restricts the creation and access entry points of the connection object, maintains the consistency of the connection state, event registration state, and message sending state, and reduces the risk of state overwriting, connection conflicts, or event listening failures when multiple threads or asynchronous tasks operate on the connection instance simultaneously.

[0052] Furthermore, in steps S2 to S8, the message pipeline processor sequentially executes message parsing, initiates asynchronous message storage, creates placeholder cards, initiates placeholder card sending, and returns a Webhook reception confirmation response. In the asynchronous processing flow after the Webhook synchronous processing ends, the message pipeline processor obtains the card location information returned by the placeholder card sending operation and performs prompt word injection. The streaming response bridging processor generates a bridging association identifier for this large model generation process, establishes a bridging relationship between the bridging association identifier and the card location information, and completes the large model streaming response event subscription through the event dispatcher, thus completing the bridging relationship establishment and event subscription. Subsequently, the message pipeline processor starts the intelligent agent executor to execute the large model generation task. By segmenting the Webhook synchronous processing flow and the background asynchronous processing flow, the Webhook interface only undertakes message reception, basic parsing, asynchronous task initiation, and receiving confirmation responses, while the waiting for placeholder card location information, prompt word construction, bridge relationship establishment, event subscription, and large model invocation are transferred to the background asynchronous process for execution. This can shorten the Webhook interface occupation time and ensure that the large model has a complete context for receiving streaming events and locating target cards before it starts, preventing the loss of streaming response events or writing to incorrect cards due to the reversed processing sequence.

[0053] Furthermore, in step S3, after initiating the asynchronous message storage operation, the message pipeline processor continues to execute the placeholder card creation and placeholder card sending operations without waiting for the asynchronous message storage operation to complete. This allows the message storage process and the placeholder card processing process to be executed asynchronously. Through the asynchronous message storage setting, the input / output waiting time generated by the user message being written to the database, log system, or other message storage units no longer blocks the placeholder card creation and Webhook confirmation response process. The message storage task and the placeholder card sending task can proceed in parallel, thereby reducing the time for the user to wait for platform confirmation and placeholder card display after sending a message. At the same time, the message storage operation can still be completed independently in the background, taking into account the needs of message retention, session tracing, and fault diagnosis.

[0054] Furthermore, in steps S4 to S8, the placeholder card includes at least a response content display area. This area is written with waiting response information when the placeholder card is created. After initiating the placeholder card sending operation, the message pipeline processor generates and returns a Webhook confirmation response. After the Webhook synchronization processing is complete, it waits for the placeholder card sending operation to finish and obtains the card location information. After obtaining the card location information, the message pipeline processor sends it to the streaming response bridge processor. The streaming response bridge processor establishes a bridge association between the bridge association identifier and the card location information and completes the large-scale streaming response event subscription. The message pipeline processor restarts the intelligent agent executor to execute the large model generation task. This prevents the streaming response events generated by the large model generation task from failing to write to the corresponding placeholder cards due to a lack of card location information. By setting the timing of sending placeholder cards first, obtaining card location information, establishing a bridging relationship and subscribing to streaming events, and finally starting the large model generation task, the enterprise instant messaging session can first display the waiting response information to the user. At the same time, it ensures that the target placeholder card has been identified and the corresponding event channel has been established before the large model outputs the first incremental response content. This solves the problem that when the large model starts up faster than the placeholder card sending speed, the early streaming response content cannot locate the target card or is directly discarded.

[0055] Furthermore, in steps S6 to S8, the message pipeline processor injects the user message content into a preset prompt word template and passes the resulting large model input prompt words to the intelligent agent executor. After the intelligent agent executor starts the large model generation task, it encapsulates the incremental response content generated during the large model generation process into a large model streaming response event and outputs it through a callback method, instead of using the complete response content generated by the large model as the synchronous return value of the Webhook interface. Through the prompt word injection setting, the user's original message can be combined with preset role information, business rules, scenario constraints, or output format requirements into a structured large model input prompt word, enabling the intelligent agent executor to execute the generation task according to the preset business scenario of the enterprise instant messaging robot and output incremental response content through a callback method. This allows the large model to generate a corresponding streaming response event as soon as a segment of content is generated, without waiting for the complete response generation to finish, thereby reducing the display delay of the first segment of content and separating the synchronous return process of the Webhook interface from the long generation process of the large model.

[0056] Furthermore, in steps S7 and S8, the bridging association identifier is generated before the large model generation task starts, and is used to uniquely correspond to this large model generation process. When the intelligent agent executor starts the large model generation task, it receives the bridging association identifier and writes the bridging association identifier into the task call parameters or the streaming callback context identifier. When the intelligent agent executor returns the large model generation task identifier, it establishes a correspondence between the large model generation task identifier and the bridging association identifier. The card positioning information includes at least one of the following: placeholder card identifier, instant messaging message identifier, or card update address. After receiving the large model streaming response event, the event dispatcher determines the appropriate identifier based on the bridging association identifier, the large model generation task identifier, or the streaming callback context identifier. At least one of the following queries the corresponding card location information, and the large model streaming response event is handed over to the streaming response bridge processor associated with the corresponding placeholder card. Through the multi-level mapping settings between the bridge association identifier, the large model generation task identifier and the card location information, the large model generation task can establish a card correspondence using the pre-generated bridge association identifier before the platform task identifier is returned. After obtaining the large model generation task identifier, the task mapping relationship is supplemented. When multiple users, multiple sessions or multiple large model generation tasks run concurrently, the event dispatcher can locate the unique generation process and placeholder card according to any valid identifier carried by the event, avoiding the incremental response content of different tasks from being cross-streamed, spliced, or written to other session cards.

[0057] Furthermore, in steps S9 and S10, the incremental response content is the text content output in stages during the generation of the large model. Each time a large model streaming response event is received, the streaming response bridging processor concatenates the incremental response content to the cumulative response content of the corresponding large model generation process according to the receiving order of the streaming response events or the event sequence number carried by the streaming response events. Based on the card positioning information and the updated cumulative response content, a card update request is generated, and the cumulative response content is written to the same placeholder card via the card PUT update method. After the large model generation task is completed, the final cumulative response content is written to the placeholder card, the subscription to the large model streaming response event corresponding to this large model generation process is released, and the bridging association identifier, the large model generation task identifier, and the card positioning information are deleted. The system establishes the correspondence between events and releases the corresponding cumulative response content cache. By accumulating incremental response content according to the event reception order or event sequence number order, it can maintain the generation order of text content when streaming events arrive continuously or when there is a short asynchronous delay. By using the card PUT update method, all current cumulative response content is written into the same placeholder card, so that each card update is based on the complete cumulative state. Even if an update request fails, the current cumulative response content can be used to overwrite the update during subsequent retries, reducing the situation where incomplete card display content is caused by the loss of local incremental data. By unsubscribing from events, deleting the identifier mapping, and releasing the cumulative response cache after the task ends, it can prevent the completed task from continuing to receive events or occupying memory resources, while avoiding the influence of historical mapping relationships on the card positioning of subsequent new tasks.

[0058] Example 1: System Overall Architecture

[0059] like Figure 1 As shown, this embodiment includes an enterprise instant messaging platform, a Webhook interface, a message pipeline processor, a message storage unit, a streaming response bridge processor, a global singleton connection service, an intelligent agent executor, and a large model.

[0060] An enterprise instant messaging platform includes a user client or session window, placeholder cards, and a card update interface. Users send text messages, card events, or other user messages that can trigger bot processing through the user client or session window of the enterprise instant messaging platform.

[0061] The Webhook interface is used to receive user message events pushed by the enterprise instant messaging platform, and after the message pipeline processor initiates the placeholder card sending operation, it returns a Webhook reception confirmation response to the enterprise instant messaging platform.

[0062] The message pipeline processor includes message parsing, asynchronous message storage triggering, placeholder card processing, prompt word injection, and agent task startup functions. The message parsing function is used to obtain session information, user information, and message content from user messages. The asynchronous message storage triggering function is used to asynchronously write user messages to the message storage unit. The placeholder card processing function is used to create placeholder cards and initiate the sending of placeholder cards. The prompt word injection function is used to combine user message content with preset prompt word templates. The agent task startup function is used to start the agent executor after the bridging relationship and streaming event subscription are established.

[0063] The message storage unit is used to asynchronously save user messages, session information, and data related to the current message processing. After initiating an asynchronous message storage operation, the message pipeline processor continues to execute the placeholder card creation and sending operations without waiting for the message storage unit to return the storage result.

[0064] The streaming response bridging processor includes functions for generating bridging association identifiers, establishing bridging relationships, subscribing to streaming response events, querying card location information, maintaining cumulative response content, generating card update data, and cleaning up after task completion.

[0065] The global singleton connection service includes WebSocket persistent connections, an event dispatcher, and connection startup state management. The WebSocket persistent connection is used to maintain a continuous connection with the enterprise instant messaging platform. The event dispatcher is used to register event handlers with the enterprise instant messaging platform, receive large-scale streaming response events, and distribute streaming response events to streaming response bridge handlers. The connection startup state management function is used to record whether the singleton connection service has been started.

[0066] In one specific implementation, the global singleton connection service can be implemented using the WsService class. The WebSocket connection instance, event dispatcher, and startup flag are all encapsulated through private fields. The startup flag can be represented by #started. When the startup interface is called, the status of #started is first checked. If #started is in an unstarted state, a WebSocket long connection is established and the platform event handler is registered. Then, #started is updated to the started state. If #started is already in the started state, the startup operation is terminated directly.

[0067] The WsService class name and #started field name mentioned are only one code implementation method and do not constitute a limitation on the scope of protection of this invention.

[0068] The agent executor receives input prompts for the large model, creates and initiates the large model generation task. The agent executor passes the bridging association identifier as a task invocation parameter or a streaming callback context identifier to the large model invocation process.

[0069] The large model is used to generate response content based on input prompts, and continuously outputs text fragments generated during the process as incremental response content. The agent executor encapsulates the incremental response content, along with bridging association identifiers, large model generation task identifiers, or streaming callback context identifiers, into large model streaming response events.

[0070] Example 2: Message Reception and Fast Webhook Response

[0071] like Figure 2 As shown, first, step S1 is executed to start the global singleton connection service, establish a WebSocket long connection, and register the enterprise instant messaging platform event handler with the event dispatcher.

[0072] The Webhook interface and the WebSocket long connection have different functions. The Webhook interface serves as the user message entry point, used to receive user messages actively pushed by the enterprise instant messaging platform and quickly return a confirmation response. The WebSocket long connection is used to continuously maintain the connection state between the enterprise instant messaging robot and the enterprise instant messaging platform, listen for connection status events and other asynchronous platform events, and provide a connection foundation for the event dispatcher to work continuously.

[0073] In step S2, the Webhook interface receives user messages, parses the user messages, and obtains session information, user information, and message content. The session information is used to determine the enterprise instant messaging session to which the placeholder card needs to be sent, the user information is used to determine the message sender or to perform identity verification, and the message content is used to build input prompts for the large model.

[0074] In step S3, the message pipeline processor calls the asynchronous write interface of the message storage unit to initiate a user message storage operation. After initiating asynchronous storage, the message pipeline processor does not wait for the storage result.

[0075] In step S4, the message pipeline processor creates a placeholder card based on the session information and initiates a placeholder card sending operation. The response content display area of ​​the placeholder card can be pre-written with messages such as "Generating a response, please wait".

[0076] After executing step S5 and initiating the placeholder card sending operation, the Webhook interface immediately returns a receipt confirmation response to the enterprise instant messaging platform, ending the current Webhook synchronous processing. The placeholder card sending operation, bridging relationship establishment, and large model generation tasks all continue to be executed in the background asynchronous processing flow after the Webhook synchronous processing ends.

[0077] Using the above method, even if generating answers for a large model takes a long time, the Webhook interface will not continuously occupy the synchronous processing thread.

[0078] Example 3: Card Location Information Acquisition and Bridging Relationship Pre-establishment

[0079] After the placeholder card is sent to the enterprise instant messaging session, the enterprise instant messaging platform returns the card location information. The card location information can be the placeholder card identifier, the instant messaging message identifier, the card update address, or the card object returned by the corresponding platform software development kit.

[0080] The message pipeline processor waits for the placeholder card sending operation to return card location information. This waiting process is in a background asynchronous process and does not affect the already completed receive confirmation response from the Webhook interface.

[0081] After obtaining the card location information, the message pipeline processor combines the user message content with the preset prompt word template to form a large model input prompt word, and sends the card location information to the streaming response bridge processor.

[0082] The streaming response bridging processor generates a bridging association identifier for this large model generation process. The bridging association identifier is generated before the large model generation task starts and does not depend on the large model generation task identifier. It can be a universally unique identifier, a random identifier, a message processing serial number, or a combination of session serial number and time information.

[0083] The streaming response bridging processor establishes a bridging relationship between the bridging association identifier and the card location information. This bridging relationship can be stored in a memory-mapped structure, a cache unit, a database, or a streaming callback context object.

[0084] After the bridging relationship is established, the large model streaming response events output by the agent executor are subscribed to through the event dispatcher. Thus, even if the large model outputs its first streaming response event immediately after startup, the corresponding card location information can be queried based on the bridging association identifier.

[0085] Example 4: Large Model Task Launch and Task Identifier Association

[0086] After establishing the bridging relationship and subscribing to streaming response events, the message pipeline processor starts the agent executor to perform the large model generation task.

[0087] When starting a large model generation task, the bridging association identifier is passed to the agent executor as a task call parameter or a streaming callback context identifier.

[0088] In some large model invocation frameworks, the large model generation task identifier needs to be returned by the agent executor after the task is created. At this time, after obtaining the large model generation task identifier, the correspondence between the large model generation task identifier and the bridging association identifier is further established.

[0089] Therefore, subsequent streaming response events can locate the corresponding placeholder card using any of the following information:

[0090] Bridge association identifier;

[0091] Large model generates task identifiers;

[0092] Streaming callback context identifier;

[0093] The large model generates a combination of task identifiers and bridging association identifiers.

[0094] The bridging association identifier and the large model generation task identifier are distinguished from each other to avoid logical conflicts such as requiring the large model generation task identifier to be obtained in advance when the large model task has not yet started.

[0095] Example 5: Streaming Response Event Bridging and Cumulative Response Calculation

[0096] like Figure 3 As shown, the large model outputs incremental response content in stages during the process of generating response content. The agent executor encapsulates the incremental response content and at least one of the following: bridging association identifier, large model generation task identifier, or streaming callback context identifier, into a large model streaming response event.

[0097] After receiving a large model streaming response event, the event dispatcher distributes the large model streaming response event to the streaming response bridge processor. The streaming response bridge processor queries the bridging relationship based on the information carried by the large model streaming response event, obtains the corresponding card positioning information, and determines the large model generation process to which this large model streaming response event belongs.

[0098] The streaming response bridge processor maintains the cumulative response content for each large model generation process.

[0099] Let the streaming response bridge processor be the first The content of the incremental response received this time is , No. The cumulative response content saved after streaming event processing is ,in If is a positive integer greater than or equal to 1, then the th... The cumulative response content after secondary stream response event processing is as follows:

[0100]

[0101] in:

[0102]

[0103] In the formula, Indicates the first The cumulative response content generated after secondary stream response event processing Indicates the first The content of the received incremental response, symbol This indicates a text concatenation operation performed according to the order in which streaming response events are received or the order of their sequence numbers. This indicates that the cumulative response content was empty before the large model generation task started.

[0104] Upon receiving the first incremental response:

[0105]

[0106] When the second incremental response content is received:

[0107]

[0108] This process continues until the large model generation task is completed.

[0109] When a streaming response event does not carry an event sequence number, the event dispatcher performs serial distribution for streaming response events corresponding to the same bridging association identifier, and the streaming response bridging processor splices the incremental response content according to the receiving order of the streaming response events.

[0110] When a streaming response event carries an incrementing event sequence number, the streaming response bridge processor determines the splicing order of each incrementing response content based on the event sequence number.

[0111] Each time a new cumulative response is generated, the streaming response bridge processor generates card update data based on the card location information and the updated cumulative response, and calls the card update interface to write the cumulative response to the same placeholder card.

[0112] When the card update interface call fails, the streaming response bridge processor retains the current cumulative response content and regenerates the card update request according to the preset retry rules. Subsequent received incremental response content is then appended to the current cumulative response content, so that it is not lost due to a single card update failure.

[0113] Therefore, the cumulative response content is maintained independently in the streaming response bridge processor, and its formation process does not depend on whether the card update interface is successfully called.

[0114] Example 6: Card Update and Cumulative Content Reload

[0115] After each update of the cumulative response content, the streaming response bridge processor generates card update data based on the updated cumulative response content. The card update data includes all the cumulative response content that the placeholder card needs to display, rather than just the single incremental response fragment carried by this streaming response event.

[0116] A card update request is generated based on the card location information and card update data. The card update interface provided by the enterprise instant messaging platform is then called to write the updated cumulative response content to the same placeholder card.

[0117] In one specific implementation, the card update interface uses the PUT interface. Therefore, each PUT operation writes all the response content that has been accumulated so far, thereby avoiding the overall overwrite PUT interface from overwriting the previously displayed content.

[0118] In other embodiments, the card update interface may also be a PATCH interface, a message editing interface, a card update method, or an update interface encapsulated by a platform software development kit, and the present invention does not limit this to any particular type.

[0119] Before the large model generation task is finished, the streaming response bridge processor continues to receive the next large model streaming response event and repeatedly executes the process of querying card location information, splicing cumulative response content, generating card update data, and updating placeholder cards.

[0120] When the large model generation task is completed, the streaming response bridge processor writes the final accumulated response content to the placeholder card and switches the card display status from the generation in progress state to the generation completed state.

[0121] Example 7: Task Completion and Resource Release

[0122] Upon receiving the event indicating the large model generation task has ended, the streaming response bridge processor stops receiving incremental response content corresponding to the current generation process.

[0123] The streaming response bridge processor writes the final accumulated response content to the corresponding placeholder card and unsubscribes from the streaming response event subscription for the current large model generation process.

[0124] After the event subscription is unsubscribed, delete the correspondence between the bridging association identifier and the card location information; if a correspondence between the large model generation task identifier and the bridging association identifier has already been established, delete that correspondence at the same time.

[0125] After completing the mapping deletion, release the cached cumulative response content and temporary state data corresponding to this large model generation process.

[0126] By unsubscribing, deleting mappings, and releasing caches after a task is completed, it is possible to prevent the continuous accumulation of bridging mappings and cumulative response caches during long-term operation of enterprise instant messaging robots.

[0127] Example 8: Concurrent Processing of Multiple Messages

[0128] When multiple users or the same user sends multiple messages consecutively, each user message creates a corresponding large model generation process and generates different bridging association identifiers.

[0129] Different bridging association identifiers correspond to their respective card location information, streaming event subscription relationships, and cumulative response content, enabling the event dispatcher to send streaming response events generated by different large model generation tasks to the corresponding streaming response bridging processors.

[0130] Since the global singleton connection service maintains the WebSocket connection and event dispatcher uniformly, it is not necessary to re-establish the WebSocket connection for each user message.

[0131] Different messages are isolated by bridging association identifiers, which can prevent incremental response content of a large model generation process from being written into placeholder cards corresponding to another large model generation process.

[0132] Working principle: First, a global singleton connection service is started. This service uses a startup flag to determine if the WebSocket long connection and the enterprise instant messaging platform's event handler have been started. If not, a WebSocket long connection is established and event registration is completed. If already started, duplicate connection establishment and event registration are prevented. After a user sends a message through the enterprise instant messaging platform's session window, the Webhook interface receives and parses the message, submitting the parsing result to the message pipeline handler. The message pipeline handler initiates an asynchronous message storage operation, creates a placeholder card without waiting for storage to complete, and then initiates the placeholder card sending operation. Subsequently, the Webhook... The OK interface immediately returns a receipt confirmation response to the enterprise instant messaging platform, ending the synchronous processing. In the background asynchronous processing flow, after the placeholder card is sent, card location information is returned. The message pipeline processor obtains the card location information, combines the user message content with the preset prompt word template to form the input prompt word for the large model, and sends the card location information to the streaming response bridge processor. The streaming response bridge processor generates a bridging association identifier for this large model generation process, establishes a bridging relationship between the bridging association identifier and the card location information, and subscribes to the large model streaming response event. Afterward, the message pipeline processor starts the intelligent agent executor to call the large model and writes the bridging association identifier into the task call parameters or streaming response. When the agent executor returns the large model generation task identifier, a correspondence is established between the large model generation task identifier and the bridging association identifier. After the large model gradually generates text, the agent executor encapsulates each incremental response content into a streaming response event. The event dispatcher forwards the streaming response event to the streaming response bridging processor. The streaming response bridging processor queries the corresponding card location information based on the bridging association identifier, large model generation task identifier, or streaming callback context identifier carried by the event, and concatenates the current incremental response content into the cumulative response content of this generation process according to the event reception order or event sequence number order. Subsequently, the card update data containing all cumulative response content is used to call... The card update interface writes the cumulative response content to the same placeholder card. When the card update interface call fails, the current cumulative response content is retained and the card update request is resent according to the preset retry rules. Subsequent incremental response content continues to be concatenated to the cumulative response content. Users can see the large model response content gradually increasing in the enterprise instant messaging session until the large model generation task ends. The streaming response bridge processor writes the final cumulative response content to the placeholder card, unsubscribes from the streaming response event, deletes the correspondence between the bridge association identifier, the large model generation task identifier and the card positioning information, and releases the cumulative response content cache, thereby completing a large model asynchronous call and streaming response bridging process.

[0133] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit of this application and the scope of protection of this invention, and all of these forms are within the protection scope of this application.

Claims

1. A method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot, characterized in that, Includes the following steps: Step S1: The connection service is started, a global singleton connection service for the enterprise instant messaging platform is created, a WebSocket long connection with the enterprise instant messaging platform is established through the global singleton connection service, and the enterprise instant messaging platform event handler is registered with the event dispatcher to maintain the continuous connection state and listen for the connection status event and asynchronous platform event of the enterprise instant messaging platform. Step S2, message reception and parsing: Receive user messages sent by the enterprise instant messaging platform through the Webhook interface, parse the user messages to obtain session information, user information and message content, and submit the parsed user messages to the message pipeline processor. Step S3, asynchronous message storage: The message pipeline processor initiates the operation of asynchronously writing the parsed user message into the message storage unit, so that the message storage process does not block the response process of the Webhook interface. Step S4, placeholder card processing: The message pipeline processor creates a placeholder card for displaying the large model response content based on the session information, and initiates the operation of sending the placeholder card to the enterprise instant messaging session where the user message is located. After the placeholder card is sent, the card location information for locating the placeholder card is returned. Step S5, Webhook responds quickly. After initiating the placeholder card sending operation, it returns a Webhook reception confirmation response to the enterprise instant messaging platform, causing the Webhook interface to end this synchronization process without waiting for the placeholder card sending operation to complete or the large model to return the complete response content. Step S6, asynchronous process continuation: In the asynchronous processing process after the Webhook synchronous processing is completed, wait for the placeholder card sending operation to return the card positioning information, combine the message content with the preset prompt word template to form the large model input prompt word, and pass the large model input prompt word to the intelligent agent executor. Step S7, pre-establishment of bridging relationship: Before starting the large model generation task, the streaming response bridging processor generates a bridging association identifier that can uniquely identify the current generation process for this large model generation process, establishes the bridging relationship between the bridging association identifier and the card positioning information, and subscribes to the large model streaming response event output by the intelligent agent executor through the event dispatcher. Step S8: The large model is executed asynchronously. After the bridging relationship is established and the large model streaming response event subscription is completed, the agent executor is started to call the large model to execute the generation task, and the bridging association identifier is passed to the agent executor as a task call parameter or streaming callback context identifier. When the agent executor returns the large model generation task identifier, the correspondence between the large model generation task identifier and the bridging association identifier is established. After the generation task is started, the large model does not wait for the large model to return the complete response content. The global singleton connection service continuously maintains the WebSocket long connection, and the event dispatcher keeps the event listening state. Step S9, Streaming Response Bridging: When a large model streaming response event is received, the incremental response content and at least one of the following are obtained from the large model streaming response event: bridging association identifier, large model generation task identifier, or streaming callback context identifier. Based on the obtained information, the corresponding card positioning information is queried. The incremental response content is then concatenated to the cumulative response content of the corresponding large model generation process according to the receiving order of the streaming response event or the event sequence number order. The updated cumulative response content is then converted into card update data supported by the enterprise instant messaging platform. Step S10: Card streaming backfeed. A card update request is generated based on the card location information and card update data. The updated cumulative response content is written to the same placeholder card through the card update interface provided by the enterprise instant messaging platform. The response content generated by the large model is gradually displayed in the enterprise instant messaging session along with the streaming response event until the large model generation task is completed.

2. The method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot according to claim 1, characterized in that: In step S1, the global singleton connection service is a singleton WebSocket connection service. The singleton WebSocket connection service manages the WebSocket connection instance, event dispatcher, and connection startup status in a unified manner, so that the enterprise instant messaging robot can share the same connection service instance during operation.

3. The method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot according to claim 2, characterized in that: The singleton WebSocket connection service is set with a startup flag. Before starting the singleton WebSocket connection service, the startup flag is read. When the startup flag indicates that the singleton WebSocket connection service has not yet started, a WebSocket long connection is established, an enterprise instant messaging platform event handler is registered, and the startup flag is updated. When the startup flag indicates that the singleton WebSocket connection service has started, the repeated establishment of WebSocket long connections and the repeated registration of enterprise instant messaging platform event handlers are stopped.

4. The method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot according to claim 2, characterized in that: The singleton WebSocket connection service encapsulates the WebSocket connection instance, event dispatcher, and start flag through private fields, and controls the WebSocket connection instance through the externally provided start interface and message sending interface, so as to avoid connection state chaos caused by multiple message pipeline processors establishing WebSocket long connections separately.

5. The method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot according to claim 1, characterized in that: In steps S2 to S8, the message pipeline processor sequentially executes message parsing, initiates asynchronous message storage, creates placeholder cards, initiates placeholder card sending, and returns a Webhook reception confirmation response. In the asynchronous processing flow after the Webhook synchronous processing ends, the message pipeline processor obtains the card location information returned by the placeholder card sending operation and performs prompt word injection. The streaming response bridging processor generates a bridging association identifier for this large model generation process, establishes a bridging relationship between the bridging association identifier and the card location information, and completes the large model streaming response event subscription through the event dispatcher. After completing the bridging relationship establishment and event subscription, the message pipeline processor starts the intelligent agent executor to execute the large model generation task.

6. The method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot according to claim 1, characterized in that: In step S3, after initiating the asynchronous message storage operation, the message pipeline processor continues to execute the placeholder card creation and placeholder card sending operations without waiting for the asynchronous message storage operation to complete, so that the message storage process and the placeholder card processing process are executed asynchronously.

7. The method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot according to claim 1, characterized in that: In steps S4 to S8, the placeholder card includes at least a response content display area. When the placeholder card is created, a waiting response information is written into the response content display area. After initiating the placeholder card sending operation, the message pipeline processor generates and returns a Webhook confirmation response. After the Webhook synchronization process is completed, it waits for the placeholder card sending operation to complete and obtains the card location information. After obtaining the card location information, the message pipeline processor sends the card location information to the streaming response bridge processor. After the streaming response bridge processor establishes a bridging relationship between the bridge association identifier and the card location information and completes the subscription to the large model streaming response event, the message pipeline processor then starts the agent executor to execute the large model generation task to prevent the streaming response event generated by the large model generation task from failing to write to the corresponding placeholder card due to the lack of card location information.

8. The method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot according to claim 1, characterized in that: In steps S6 to S8, the message pipeline processor injects the user message content into the preset prompt word template and passes the resulting large model input prompt word to the intelligent agent executor. After the intelligent agent executor starts the large model generation task, it encapsulates the incremental response content generated during the large model generation process into a large model streaming response event and outputs it through a callback method, instead of using the complete response content generated by the large model as the synchronous return value of the Webhook interface.

9. The method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot according to claim 1, characterized in that: In steps S7 and S8, the bridging association identifier is generated before the large model generation task starts and is used to uniquely correspond to this large model generation process. When the intelligent agent executor starts the large model generation task, it receives the bridging association identifier and writes the bridging association identifier into the task call parameters or streaming callback context identifier. When the intelligent agent executor returns the large model generation task identifier, it establishes a correspondence between the large model generation task identifier and the bridging association identifier. The card positioning information includes at least one of the placeholder card identifier, instant messaging message identifier, or card update address. After receiving the large model streaming response event, the event dispatcher queries the corresponding card positioning information according to at least one of the bridging association identifier, large model generation task identifier, or streaming callback context identifier, and transfers the large model streaming response event to the streaming response bridge processor associated with the corresponding placeholder card.

10. The method for bridging large-scale asynchronous calls and streaming responses in an enterprise instant messaging robot according to claim 9, characterized in that: In steps S9 and S10, the incremental response content is the text content output in stages during the generation of the large model. Each time a large model streaming response event is received, the streaming response bridging processor concatenates the incremental response content to the cumulative response content of the corresponding large model generation process according to the receiving order of the streaming response events or the event sequence number carried by the streaming response events. Based on the card positioning information and the updated cumulative response content, a card update request is generated, and the cumulative response content is written to the same placeholder card through the card PUT update method. After the large model generation task is completed, the final cumulative response content is written to the placeholder card, the subscription to the large model streaming response event corresponding to this large model generation process is released, the correspondence between the bridging association identifier, the large model generation task identifier and the card positioning information is deleted, and the corresponding cumulative response content cache is released.

Images