A highly scalable and highly responsive RTC
system uses asynchronous or non-blocking I / O and HTTP response queuing to avoid
server overload. The
system distinguishes between two types of requests: an update request and a
change request. An update request is a request used to fetch an update or change notification. A
change request is a request to change some data related to the real-time session. For a given RTC session, the
server first checks whether any updates exist for an update request. If an update exists, the
server code can send an immediate response to the
client. However, if no update exists, the server code application sets a well-defined HTTP response header, and then responds. Once the server code application responds, all
application server resources are freed, and the
application server is free to
handle the next requests without blocking any thread. The existence of the well-defined response header, allows the lower level networking code to
queue the response, allowing it to be later addressed by the RTC
session ID. This way, the HTTP response can be delayed (queued) without blocking a thread in the
application server. A
Change Request is a request used to change some data related to the real-time session. Such requests would typically be tied to an action within a given RTC session (e.g., adding text to a chat, changing a slide, etc.). When these actions are sent to the server, the server-side application can determine whether the request affects the queued responses that are awaiting change notifications. In the case of a change, the notification response is computed, and then an API call into the lower-level network flushes all queued responses awaiting updates in the session. Therefore this mechanism is used to trigger the responses to complete, thus providing immediate updates to clients.