Distributed Job Processing via Autonomous Node Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed processing systems face challenges in enabling appropriate distributed processing as slave servers cannot initiate processing without allocation from the master server, leading to inefficiencies.
Innovation Solution
An information processing system where a first node creates a job list and stores it in common storage, allowing a second node to independently determine and process jobs without relying on the master server's allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a master server allocates processing to slave servers, then centralized control is maintained, but processing efficiency deteriorates because slave servers cannot start processing without allocation
Solution Approach 1:
Slave servers autonomously acquire jobs from the job queue stored in common storage without requiring allocation from the master server. Each slave server independently determines which job to process by reading from the shared job queue, enabling self-service processing that eliminates the bottleneck of centralized job allocation while maintaining system coordination through the common storage mechanism.
2Speed
If slave servers wait for master server allocation, then centralized job management is ensured, but processing speed deteriorates due to waiting time
Solution Approach 1:
The master server pre-loads and stores the job queue in common storage before slave servers need to process jobs. By preparing the job queue in advance in the shared storage location, slave servers can immediately acquire jobs without waiting for real-time allocation requests, thus reducing waiting time while maintaining centralized job management through the pre-prepared queue.
3Adaptability or versatility
If a master server manages all job allocation, then centralized control is maintained, but system flexibility deteriorates as slave servers cannot independently process jobs
Solution Approach 1:
The job management function is segmented between the master server (which maintains the job queue in common storage) and slave servers (which autonomously acquire and process jobs). This segmentation allows slave servers to independently select and process jobs from the shared queue without requiring continuous master server intervention, thereby increasing system flexibility while maintaining centralized job definition through the common storage mechanism.
Data Source
AI summary
An information processing system according to the present application is an information processing system including a plurality of nodes including a first node and a second node that execute distributed processing, in which the first node creates a job list including a plurality of jobs into which information processing requested to be processed in the information processing system is divided, and stores the created job list into a common storage accessible to each of the plurality of nodes, and the second node refers to the common storage, determines a targeted job being a job in the job list that is to be set as a processing target, and updates the job list after processing of the targeted job.