Inflight GUI Streaming With Local Interactive Response
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for remote operation of interactive programs in safety-critical environments, such as aircraft cockpits, suffer from high latency due to user interactions being communicated back and forth between remote and onboard systems, leading to poor responsiveness.
Innovation Solution
A system that executes a program on a remote system, generates a data stream with primary and supplemental information for a graphical user interface (GUI), and modifies the GUI response locally based on user interactions without immediate communication with the remote system, using embedded supplemental data to enhance responsiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If user interactions are communicated back and forth between remote and onboard systems, then program execution remains centralized on remote system, but system responsiveness deteriorates due to high latency
Solution Approach 1:
The system divides the GUI into two parts: a static base displayed remotely and an interactive overlay generated locally. This segmentation allows the remote system to maintain centralized execution while the local system handles interactive responses independently, eliminating latency for user interactions.
Solution Approach 2:
A local intermediary system is introduced between the remote system and the display. This intermediary receives the base GUI, adds interactive overlays locally, and combines them for display. The intermediary enables fast local responses without requiring constant communication with the remote system.
2Extent of automation
If interactive elements require remote system processing, then security and control remain centralized, but user experience deteriorates due to slow response
Solution Approach 1:
The system segments control functions: the remote system maintains centralized control of the base GUI and program logic, while the local system handles interactive element responses. This allows centralized automation to remain intact while local responsiveness is achieved through independent overlay generation.
Solution Approach 2:
The remote system pre-generates the base GUI and makes it available to the local system in advance. Interactive overlays are prepared and embedded in the data stream before the user interacts, allowing immediate local response without waiting for remote processing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods are provided that allow for providing interactive functionality to streamed data. The systems include a remote system configured to execute a program, generate a data stream that includes primary information indicative of a visual representation of a graphical user interface (GUI) of the program, and transmit the data stream. The system includes an aircraft that includes a controller configured to receive the data stream from the remote system, display the visual representation of the GUI on a display system based on the primary information, detect an interaction by a user with an interactive element of the GUI represented on the display system, and modify the visual representation of the GUI on the display system in response to detecting the interaction such that a modified visual representation of the GUI is displayed on the display system using supplemental information embedded in the data stream.