Distributed Publish-Subscribe Messaging with Schema Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
IoT devices generate large amounts of unstructured data that are difficult to manage and integrate across disparate systems, leading to inefficiencies in data communication and processing.
Innovation Solution
A publish-subscribe messaging method that structures raw data using schemas and distributes them across a network, enabling transformation and rule execution at optimal nodes for improved performance and data consistency, while also providing a marketplace for on-demand data processing and rule execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If raw data from IoT devices is transmitted and processed in a centralized manner, then data integration and management are simplified, but processing time and network bandwidth consumption increase
Solution Approach 1:
The patent segments the centralized data processing function into distributed processing nodes deployed across the network. Each node independently processes data locally, eliminating the single-point bottleneck of centralized processing. This segmentation allows parallel processing of multiple data streams simultaneously, reducing overall processing time while maintaining simplified data management through standardized processing templates.
Solution Approach 2:
The patent introduces a new dimension of processing by deploying processing nodes at multiple network locations rather than consolidating all processing in one central location. This spatial distribution across the network dimension enables local processing that reduces network bandwidth consumption and processing time, while the standardized processing templates maintain operational simplicity.
2Productivity
If data processing is distributed across multiple nodes, then processing speed and network efficiency improve, but system complexity increases
Solution Approach 1:
The patent creates universal processing templates that can be deployed across multiple processing nodes throughout the network. These templates encapsulate the processing logic, making each node multi-functional and capable of handling various data types with the same standardized approach. This universality increases processing speed through parallel distribution while minimizing system complexity by reusing the same templates across all nodes.
Solution Approach 2:
The patent changes the parameter of processing location from centralized to distributed across multiple nodes. By modifying this spatial parameter and implementing standardized processing templates at each node, the system achieves higher processing speed through parallel operations while the standardization aspect keeps system complexity manageable through reusable configurations.
3Reliability
If schemas are enforced at every processing node, then data consistency and integrity are ensured, but processing overhead and computational resources increase
Solution Approach 1:
The patent applies schemas to data templates in advance before data processing occurs. This preliminary action pre-validates the data structure requirements, so that during actual processing, nodes only need to check against the pre-defined schema rather than performing full validation. This ensures data consistency and integrity while significantly reducing computational overhead and energy consumption during runtime processing.
Solution Approach 2:
The patent implements partial schema validation by applying schemas only to the necessary data fields and only at the points where data is generated or transformed. Rather than enforcing complete schema validation at every processing stage, this partial approach maintains data consistency where critical while minimizing unnecessary computational resources and energy consumption at other stages.
4Quantity of substance
If more processing nodes are deployed to handle increased data volume, then data processing capacity increases, but network infrastructure cost and device complexity increase
Solution Approach 1:
The patent employs universal processing templates that can be instantiated on standard network devices, making each device multi-functional for data processing. This approach increases data processing capacity by utilizing existing network infrastructure rather than requiring specialized hardware, thereby avoiding increased network infrastructure costs while still achieving the needed processing capacity through template-based standardization.
Data Source
AI summary
Technology for a publish-subscribe messaging method may include determining a schema for structuring raw data published in a publication by a publisher in a publish-subscribe system at a channel node in a network and distributing the schema from the channel node to other channel nodes in the network. The method may include identifying the publication from the publisher at one of the channel nodes and applying the schema to the raw data of the publication at the channel node, transforming the raw data to transformed data. The transformed data may be provided at a subscriber in the publish-subscribe system.