Edge Indexing of Streaming Sensor Data for Real-Time Local Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud computing architectures face challenges in latency, bandwidth usage, data privacy, and the capacity to process large volumes of data in real-time for applications like AI and ML, particularly in remote environments where cloud connectivity is limited.
Innovation Solution
Implementing edge computing systems that process, store, and index streaming data locally using edge devices equipped with machine learning networks to perform real-time search and retrieval without cloud connectivity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If data is transmitted to centralized data centers for processing, then processing capacity is improved, but latency increases and bandwidth usage increases
Solution Approach 1:
The patent divides the centralized processing architecture into distributed edge computing nodes that process data locally. Each edge device segments the data processing workload, performing computations closer to the data source rather than transmitting all data to centralized data centers. This segmentation reduces latency while maintaining processing capacity through parallel distributed processing.
Solution Approach 2:
The patent introduces a new architectural dimension by deploying computing resources at the network edge rather than only at centralized cloud data centers. This spatial dimensionality change enables data processing to occur at multiple locations simultaneously, reducing the distance data must travel and thereby reducing latency while preserving processing capacity.
2Power
If data is transmitted to centralized data centers for processing, then processing capacity is improved, but bandwidth usage increases
Solution Approach 1:
The patent segments the data processing function across distributed edge devices, so that only processed results or essential data need to be transmitted to centralized systems. This eliminates the need to transmit large volumes of raw data over the network, significantly reducing bandwidth consumption while maintaining processing capacity through distributed computation.
Solution Approach 2:
The patent extracts the computing function from centralized data centers and places it at the network edge. By taking out the processing capability from the centralized architecture and distributing it to edge devices, the system reduces network bandwidth usage since data no longer needs to be transmitted to and from centralized data centers for processing.
3Loss of time
If data is processed locally at the edge, then latency is reduced and bandwidth usage is reduced, but processing capacity is limited
Solution Approach 1:
The patent segments the processing architecture into multiple distributed edge nodes that can operate independently and in parallel. Each edge device provides local processing capacity, and collectively these segmented nodes deliver aggregate processing power that can handle large volumes of data while maintaining low latency through local processing.
Solution Approach 2:
The patent creates a universal edge computing platform that can perform multiple functions: local data processing, filtering, preliminary analysis, and selective data transmission to centralized systems. This multi-functional edge device architecture ensures that processing capacity is not limited but rather enhanced through the versatility of edge computing nodes.
4Reliability
If data is stored and processed at the edge, then data privacy is improved, but device complexity increases
Solution Approach 1:
The patent segments the system architecture to separate data storage and processing functions at the edge from centralized management and coordination functions. This segmentation allows edge devices to maintain data locally for privacy while the segmented management layer handles complexity through distributed coordination protocols and standardized interfaces.
Solution Approach 2:
The patent introduces intermediary layers and standardized communication protocols that mediate between the simple local edge processing functions and the complex requirements of distributed system management. These intermediaries abstract the complexity, allowing edge devices to maintain data privacy locally while coordinating with the broader system through standardized interfaces.
Data Source
AI summary
Systems and techniques are provided for processing streaming data. For example, a process can include obtaining, by an edge device, a plurality of data streams corresponding to objects, activities, or events registered in an edge environment associated with the edge device. One or more machine learning networks implemented on the edge device can generate a plurality of features and/or embeddings corresponding to each respective data stream of the plurality of data streams. The edge device can determine a subset of salient content from the plurality of data streams, based on analyzing the plurality of features and/or embeddings generated for each respective data stream. The edge device can generate index information corresponding to the determined subset of salient content. The subset of salient content and the generated index information can be stored locally at the edge device for search and retrieval based on local queries received at the edge device.