Persistent TCP Connection for Real-Time Data Presentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional architectures for real-time, low-latency applications that involve multiple writes and queries simultaneously do not scale well, particularly when the data arrival rate differs from the request rate, leading to scalability issues in visualizing sensor data.
Innovation Solution
Establishing persistent TCP network connections between processes to dynamically allocate memory for static and dynamic data, allowing a Web server to access and present updated data through a Web service interface, enabling efficient data presentation without the need for relational databases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional relational database architecture is used for data service, then data can be stored and queried, but scalability deteriorates when multiple writes and queries run simultaneously with real-time requirements
Solution Approach 1:
The system segments the data service architecture into separate components: a data reception process that handles incoming sensor data, a web server process that handles client requests, and a shared memory space for data storage. This segmentation allows simultaneous writes and queries to occur without blocking each other, improving scalability while maintaining reliability.
Solution Approach 2:
Shared memory acts as an intermediary between the data reception process and the web server process. Instead of direct database queries blocking data ingestion, the intermediary memory buffer allows both operations to proceed concurrently, resolving the scalability issue while preserving data integrity.
2Stability of the object's composition
If data is written into database before querying, then data persistence is ensured, but latency increases for real-time applications
Solution Approach 1:
The system performs preliminary action by pre-allocating shared memory spaces before data arrives. When sensor data comes in, it can be immediately placed in the pre-prepared memory buffer and simultaneously served to clients, eliminating the time-consuming sequence of write-then-query operations in traditional databases.
Solution Approach 2:
The shared memory architecture enables continuous data flow from sensors to clients without interruption. Data is continuously received, stored in memory, and continuously served to multiple clients simultaneously, maintaining persistent availability without the start-stop nature of traditional database transactions.
3Ease of operation
If traditional web browser architecture is used to access sensor data, then user interface presentation is achieved, but data rate mismatches cause inefficiency
Solution Approach 1:
The shared memory buffer serves multiple functions simultaneously: it acts as a data reception point from sensors, a storage medium for persistent data, a queue for data transmission, and a direct access source for multiple web clients. This multi-functionality handles varying data rates efficiently while maintaining ease of data presentation through standard web interfaces.
Data Source
AI summary
A method of presenting data over a Web service interface includes: establishing, by a first computer process, a persistent transmission control protocol (TCP) network connection between the first computer process and a second computer process; dynamically allocating, by the second computer process, memory in response to receipt of static data over the persistent TCP network connection from the first computer process; updating, by the second computer process, the memory in response to receipt of dynamic data received over the persistent TCP network connection from the first computer process; and enabling, by the second computer process, a Web server to access the updated data for presentation by the Web service interface. The static data identifies a given entity and the dynamic data includes metric data provided for the entity.