Distributed Data Distribution with Resource Manager
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data distribution systems face issues such as server overload, unreliable communication, scalability challenges, and inefficiencies due to ignorance of global resource constraints, leading to decreased throughput and prioritization difficulties when distributing large volumes of data across widely dispersed networks.
Innovation Solution
A system with multiple masters and a resource manager that prioritizes data sources based on network distance and idleness, allocates resources efficiently, and schedules copy operations to optimize data transmission, utilizing slave state information to determine file requirements and resource availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple clients simultaneously request the same file from the same server, then the server can fulfill client requests, but the server becomes overloaded and cannot respond efficiently to normal requests
Solution Approach 1:
The patent segments the centralized server architecture into a distributed peer-to-peer network where multiple nodes can serve as sources. Instead of relying on a single server, the system divides the file distribution task across multiple peers, allowing any peer with the file to act as a source for multiple clients simultaneously, thereby preventing any single node from becoming overloaded.
Solution Approach 2:
The patent introduces a coordinator as an intermediary that manages peer selection and resource allocation. The coordinator receives file requests, identifies appropriate peer sources based on current network conditions and resource availability, and directs clients to optimal sources. This intermediary layer distributes the serving burden across multiple peers rather than concentrating it on a single server.
2Area of stationary object
If data is distributed between widely dispersed data sites, then network coverage is expanded, but communication consumes valuable system resources and becomes slower due to large distances
Solution Approach 1:
The patent implements local quality by allowing peers to serve files locally to nearby clients rather than all clients contacting a centralized server or distant peers. The system prioritizes selecting source peers that are geographically or topologically close to requesting clients, reducing transmission distance and resource consumption. This local-serving approach maintains expanded network coverage while minimizing the energy and resources required for data transmission.
3Quantity of substance
If the number of machines and data sites increases within a network, then network capacity and coverage improve, but scalability becomes an issue as centralized control becomes overwhelmed and vulnerable to failure
Solution Approach 1:
The patent segments the centralized control function into a distributed architecture where each peer runs its own control logic and the coordinator manages only high-level resource allocation. Instead of a single centralized controller handling all decisions for every peer, the system divides control responsibilities across multiple independent nodes, allowing the network to scale without overwhelming any single control entity.
Solution Approach 2:
The patent implements self-service by enabling peers to autonomously manage their own file sharing operations and resource allocation decisions. Each peer independently monitors its own resource status, responds to requests from the coordinator, and makes local decisions about file transfers. This self-managing capability eliminates the need for complex centralized control mechanisms that would become overwhelmed as the network grows.
4Adaptability or versatility
If peer-to-peer distribution systems are used to address scalability, then network growth is enabled, but global resource constraints are ignored leading to decreased aggregate throughput due to collisions and packet drops
Solution Approach 1:
The patent implements feedback mechanisms where the coordinator continuously monitors network conditions, resource availability, and transfer performance. Based on this feedback, the coordinator dynamically adjusts peer selections, allocates resources, and manages transfer priorities. This feedback loop ensures that peer-to-peer operations are coordinated to avoid collisions and packet drops, maintaining high aggregate throughput while the network scales.
Solution Approach 2:
The patent introduces dynamic resource allocation and peer selection that adapts to changing network conditions. Instead of static peer assignments, the system continuously evaluates resource availability, network load, and transfer performance to dynamically adjust which peers serve which clients. This dynamic approach optimizes aggregate throughput by preventing resource conflicts and adapting to network changes as the system scales.
Data Source
AI summary
A system having a resource manager, a plurality of masters, and a plurality of slaves, interconnected by a communications network. To distribute data, a master determined that a destination slave of the plurality slaves requires data. The master then generates a list of slaves from which to transfer the data to the destination slave. The master transmits the list to the resource manager. The resource manager is configured to select a source slave from the list based on available system resources. Once a source is selected by the resource manager, the master receives an instruction from the resource manager to initiate a transfer of the data from the source slave to the destination slave. The master then transmits an instruction to commence the transfer.