Metadata Storage Co-routine Segmentation for Resource Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for storing metadata in external storage devices are inefficient due to slow processor utilization and low storage speed, as they often rely on single-threaded processes that do not fully exploit computing resources, and multithreading faces issues with resource competition and data consistency.
Innovation Solution
The method employs co-routines to process metadata sequences, where key-value pairs are divided among multiple co-routines, with critical and non-critical data blocks handled separately to optimize resource utilization and enhance storage speed by leveraging available computing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If single-threaded processes are used to store metadata, then data consistency is maintained, but processor utilization is slow and storage speed is low
Solution Approach 1:
The patent segments metadata into critical and non-critical data blocks, and divides processing into multiple sequences that can be handled by different co-routines. This segmentation allows parallel processing of non-critical metadata while maintaining sequential processing for critical metadata, thereby improving storage speed without compromising data consistency.
Solution Approach 2:
The patent implements dynamic switching between single-threaded and multi-co-routine processing modes based on the type of metadata. Critical metadata is processed sequentially to ensure consistency, while non-critical metadata is processed in parallel to improve speed. This dynamic approach optimizes processor utilization while maintaining data integrity where required.
2Productivity
If multithreading is used to improve storage speed, then processor utilization increases, but resource competition and data consistency issues arise
Solution Approach 1:
The patent divides metadata into critical and non-critical segments, applying different processing strategies to each. Critical metadata that requires data consistency is processed by a single co-routine in sequential mode, while non-critical metadata is processed by multiple co-routines in parallel. This segmentation resolves the conflict between speed and consistency by applying multithreading only where it is safe.
Solution Approach 2:
The patent applies different quality levels of processing to different portions of metadata. Critical metadata receives high-quality sequential processing to ensure consistency, while non-critical metadata receives lower-quality parallel processing that prioritizes speed. This local differentiation allows the system to achieve overall improved performance without sacrificing data consistency in critical areas.
3Productivity
If computing resources are fully utilized for processing metadata, then storage speed increases, but resource competition occurs when multiple threads access shared resources
Solution Approach 1:
The patent segments the metadata processing workload into distinct sequences assigned to different co-routines. Each co-routine processes its assigned sequence independently, reducing resource competition. The system monitors computing resource availability and dynamically adjusts the number of active co-routines, optimizing resource utilization while managing complexity through structured segmentation.
Data Source
AI summary
In an example, a method comprises determining, based on a set of metadata items to be stored, a first sequence and a second sequence, wherein the first sequence and the second sequence each include a plurality of key-value pairs, each key-value pair including a metadata item from the set of metadata items and a keyword corresponding to the metadata item in the set. The method also comprises causing a first co-routine to utilize available computing resources to process the first sequence. The method further comprises in response to an amount of computing resources available for the first co-routine to process the first sequence being below a first threshold, causing a second co-routine to process the second sequence, wherein the second co-routine is different from the first co-routine. This method can increase the utilization rate of the processor and more rapidly stores the metadata into the external storage device.