Server-Side Artifact Caching for Multi-Tenant Chat Initiation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In online live chat systems, the time-consuming process of external network service calls for creating new chat sessions leads to delays, especially in multi-tenant, multi-service-instance environments, where quick chat initiation is essential.
Innovation Solution
Implementing a server-side omni-channel service that pre-creates and caches chat artifacts, allowing for rapid chat session initiation by scheduling worker service instances to perform fills based on organization configurations, including the creation of artifacts with unique identifiers, and storing these in a cache record for quick access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If chat artifacts are created on-demand through external network service calls, then system complexity is reduced, but chat session initiation time increases significantly
Solution Approach 1:
The patent pre-creates chat artifacts (visitor tokens, chat configurations, etc.) before actual chat sessions are needed and stores them in a server-side cache. When a chat session is requested, the pre-created artifacts are immediately retrieved and associated with the session, eliminating the need for time-consuming on-demand creation and achieving sub-second chat initiation while managing complexity through structured pre-processing
2Reliability
If multiple worker service instances are used to handle chat requests, then system reliability and scalability improve, but coordination complexity and resource management overhead increase
Solution Approach 1:
The patent creates multiple worker service instances that each maintain local copies of the chat artifact cache. This allows any worker instance to handle chat requests independently without complex inter-instance coordination, improving reliability through redundancy while simplifying management by avoiding the need for centralized cache synchronization between instances
3Productivity
If chat artifacts are pre-created and cached for multiple tenants, then chat initiation speed improves, but memory resource consumption increases
Solution Approach 1:
The patent implements tenant-specific chat artifact caching where each tenant's chat artifacts are stored and managed separately in the cache. This allows the system to pre-create and cache artifacts only for active or frequently-used tenants rather than all tenants, optimizing memory usage by maintaining pre-created artifacts locally for each tenant context while still achieving fast chat initiation for those tenants
Data Source
AI summary
The disclosed technology is generally directed to online chat management services. In one example of the technology, organization information is accessed for tenants. The organization information includes, for each organization, configuration information for the organization. Each organization belongs to a corresponding tenant. Worker service instances are scheduled to perform fills associated with the organizations, such that sufficient fills are performed for each organization according to the configuration information for the organizations in the accessed organization information. Each fill includes: causing artifacts associated with a function to be created, such that the artifacts associated with the function are configured based on the corresponding tenant. Artifact information associated with the created artifacts is received, including at least one unique identifier associated with the created artifacts for the function. Server-side caching of a cache record that includes the artifact information associated with the created artifacts for the function is performed.