Cache File Access Control for Multi-Process Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing methods face a trade-off between processing speed, reliability, and memory usage when managing cache files, particularly in multi-process environments, due to frequent opening and closing of cache files, which affects performance and data integrity.
Innovation Solution
A data processing apparatus determines the optimal opening and closing of cache files based on whether they can be accessed by other processes and their execution status, using a table-state management array to manage cache file areas, ensuring appropriate processing speed, reliability, and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the cache file is opened every time it is accessed, then the processing speed is improved, but the reliability deteriorates due to frequent opening and closing operations
Solution Approach 1:
The patent applies dynamics by making the cache file opening/closing behavior adaptive rather than static. The system dynamically adjusts whether to open the cache file based on real-time conditions: whether other processes are accessing the file and whether those processes are currently executing. This dynamic decision-making resolves the contradiction by opening the file only when necessary for processing speed while keeping it closed when safety is the priority.
2Productivity
If the cache file is kept open during processing, then the processing speed is improved, but the memory usage increases due to data being written to main memory
Solution Approach 1:
The patent applies local quality by making the cache file opening status file-specific rather than system-wide. Different cache files have different opening/closing states based on their individual access patterns and process requirements. This allows the system to keep only necessary cache files open in memory, minimizing memory usage while maintaining processing speed for actively accessed files.
3Reliability
If the cache file is kept open during processing, then the reliability is improved by avoiding frequent opening and closing, but the memory usage increases
Solution Approach 1:
The patent applies partial action by opening only the specific cache file that is being accessed rather than keeping all cache files open. This partial opening approach maintains reliability for the active file while minimizing memory consumption by leaving other files closed. The system performs just enough opening action to ensure reliability for current processing without the excessive memory usage of keeping everything open.
Data Source
AI summary
A data processing apparatus has a memory and a first storage apparatus and reads a data area from a second storage apparatus (a storage apparatus which is slower than the first storage apparatus and stores a plurality of data areas as data constituent elements). Regarding each processing block, the data processing apparatus determines whether an open/closed area for processing of a first processing block which is the processing block should be an entire area or a partial area of the data area, based on whether or not the data area which is input and/or output by the processing of the first processing block can also be accessed by processing of a second processing block which is a processing block other than the first processing block, and whether or not the second processing block is being executed.