Method and system for streaming transmission

Abstract

This disclosure provides a method and a system for streaming transmission. The method includes: receiving a request from a client and creating a task based on the request; invoking a language model when executing the task to continuously generate multiple data segments and storing these data segments in a streaming pool; receiving a query from the client corresponding to the task and determining whether the task is complete; in response to the task being complete, establishing a one-time connection with the client to transmit all the stored data segments; and in response to the task not yet being complete, establishing a streaming connection with the client to transmit the currently stored data segments to the client, continuously receiving subsequent data segments from the language model, and transmitting the subsequent data segments to the client via the streaming connection.

Images