Configurable Batch Processing Through Decoupled Rule Database Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hardcoded batch processing workflows are inflexible, burdensome to maintain, and computationally resource-intensive, lacking scalability and reusability.
Innovation Solution
A decoupled rule database system where a custom filtering routine intercepts a native filtering routine, retrieves additional filtering rules from an external database, and applies them to the output, allowing configuration updates without changing the underlying code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hardcoded batch processing workflows are used, then processing logic is explicitly implemented, but flexibility and maintainability deteriorate
Solution Approach 1:
The patent segments the batch processing workflow into distinct components: a fixed processing framework and configurable filtering rules. The filtering rules are stored separately in a database, allowing them to be modified independently from the core processing code. This segmentation enables maintainability by isolating configuration changes from code modifications.
Solution Approach 2:
The patent introduces dynamic configurability by allowing filtering rules to be changed at runtime without code redeployment. The system loads filtering rules from a database during execution, enabling dynamic adaptation to different processing requirements while maintaining the same underlying processing framework.
2Reliability
If hardcoded batch processing workflows are used, then processing logic is explicitly implemented, but reusability and scalability deteriorate
Solution Approach 1:
The patent creates a universal processing framework that can handle multiple different filtering rules through a single codebase. The framework is designed to load and execute various filtering rules from the database, making it multi-functional and applicable to different business scenarios without requiring separate implementations for each rule set.
Solution Approach 2:
The patent uses database-stored filtering rules as copyable configurations. Instead of hardcoding specific filtering logic, the system copies relevant filtering rules from the database into memory during execution. This allows the same processing framework to be reused across different scenarios by simply loading different rule copies.
3Reliability
If hardcoded batch processing workflows are used, then processing logic is explicitly implemented, but computational resource requirements for maintenance deteriorate
Solution Approach 1:
The patent extracts the filtering rules from the processing code and stores them in a separate database. This extraction eliminates the need to recompile and redeploy code for every filtering rule change, significantly reducing the computational resources required for maintenance. Only the lightweight database queries need to be executed when rules change.
Data Source
AI summary
This disclosure describes techniques for performing batch processing using a custom batch processing routine (e.g., a custom filtering routine) that calls a native batch processing routine (e.g., a native filtering routine) associated with a batch processing system as well as a rule database for one or more additional filtering rules. In some cases, a custom filtering routine may be configured to: (i) receive a batch of data, (ii) call a native filtering routine to process the received batch data, and (iii) apply one or more additional filtering operations on the output of the native filtering routine. For example, the custom filtering routine may be configured to: (i) make a query to the rule database to extract one or more filtering rules, and (ii) apply the extracted filtering rules to the output of the native filtering routine to perform the additional filtering operations.