Tenant Data Batch Processing via Segmented Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Information processing systems using cloud service platforms face inefficiencies in periodic processes like data totaling, where increased server numbers lead to longer processing times due to increased database queries and load, making it difficult to dynamically scale database resources without system downtime.
Innovation Solution
An information processing system that includes a database managing customer data by tenants, a queue for processing messages, and an execution system generating batch process messages for data satisfying predetermined conditions, allowing for efficient batch processing across multiple tenants, thereby reducing processing time and database load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of servers is increased to process more customer data, then processing capacity is improved, but database load increases and processing time increases
Solution Approach 1:
The patent segments customer data into different categories (e.g., by region, by service type, by data modification time) and creates separate queues for each segment. This allows parallel processing of segmented data without all servers querying the entire database, reducing database load while maintaining high processing capacity
Solution Approach 2:
The patent performs preliminary data segmentation and queue assignment before the actual processing occurs. Data is pre-categorized and placed into appropriate queues based on segmentation criteria, so that when processing begins, servers can immediately work on pre-sorted data without requiring simultaneous database queries
2Productivity
If the number of servers is increased to handle more processing requests, then processing capacity is improved, but database load increases
Solution Approach 1:
By segmenting data into multiple queues based on different criteria (region, service type, time), the patent reduces the amount of data each server needs to query from the database. Each server processes only its assigned segment, significantly reducing database load while maintaining high processing capacity
Solution Approach 2:
The patent introduces queues as intermediary structures between the database and processing servers. Instead of servers directly querying the database for all data, they retrieve pre-segmented data from queues, reducing direct database access and load
3Productivity
If batch processing is performed on multiple tenants simultaneously, then processing efficiency is improved, but system complexity increases
Solution Approach 1:
The patent segments tenants into different groups based on segmentation criteria and assigns each group to separate queues. This allows simultaneous batch processing of multiple tenant groups while maintaining clear organizational boundaries, improving efficiency without creating unmanageable complexity
Solution Approach 2:
Different queues can have different segmentation criteria tailored to specific processing needs (e.g., one queue segmented by region, another by service type). This allows optimized processing strategies for different tenant groups while maintaining overall system coherence
Data Source
AI summary
An information processing system manages data on customers using tenants which are storage regions dedicated for the customers of a database and registers a message for processing data of a processing target in a queue. The data is processed by accessing the tenants corresponding to the data so as to refer to the tenants based on the message. The information processing system registers a message for processing a plurality of data of processing targets as a batch process in the queue in a case where the plurality of data satisfy the predetermined condition when the message for processing a plurality of data of processing targets is to be registered in the queue. In the process performed on the plurality of data, the different tenants are accessed to be referred to.