Database Proxy Prepared Statement Caching for High-Concurrency Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed database preprocessing is inefficient due to high resource consumption and increased load on databases from frequent delivery of prepared statements in high concurrency scenarios.
Innovation Solution
A database proxy layer caches session identifiers for prepared statements, allowing processing based on cached session identifiers to reduce direct interactions with the database, thereby optimizing performance and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data preprocessing is performed on each node independently in a distributed database, then system scalability and fault tolerance are improved, but data consistency and accuracy deteriorate due to lack of coordination
Solution Approach 1:
A coordination mechanism is introduced as an intermediary between independent node preprocessing operations. This coordinator tracks preprocessing states across nodes and manages the overall process, ensuring that independent node operations maintain data consistency through coordinated communication and state management.
2Measurement precision
If complex data preprocessing operations are performed, then data quality and accuracy are improved, but processing time and system complexity increase
Solution Approach 1:
Complex data preprocessing operations are divided into independent segments that can be executed concurrently on different nodes. Each node performs its assigned preprocessing task independently, and the coordination mechanism manages the overall process, enabling parallel execution that reduces total processing time while maintaining data quality.
Solution Approach 2:
The system performs preliminary actions by establishing the coordination mechanism and pre-defining preprocessing tasks before actual data processing begins. This allows for optimized task allocation and execution planning, reducing overall processing time while ensuring high data quality through coordinated control.
3Measurement precision
If centralized data preprocessing is implemented, then data consistency is improved, but system complexity and single point of failure risk increase
Solution Approach 1:
The centralized preprocessing function is segmented and distributed across multiple nodes in the system. Each node executes preprocessing operations independently, eliminating the single point of failure while maintaining data consistency through the coordination mechanism that manages distributed state tracking and synchronization.
4Reliability
If independent node preprocessing is used, then fault tolerance is improved, but coordination overhead and communication cost increase
Solution Approach 1:
The coordination mechanism serves as an intermediary that manages communication and coordination between independent nodes. It tracks preprocessing states, manages task allocation, and coordinates data synchronization, thereby enabling fault tolerance through independent node operation while controlling coordination overhead through efficient state management.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Disclosed are a distributed database preprocessing method, a database proxy layer, a system and a storage medium. The distributed database preprocessing method, applied to a database proxy layer, includes: receiving (S101) a request corresponding to a prepared statement; and performing processing (S102) according to the request corresponding to the prepared statement and a session identifier cached in a prepared statement cache library, a session corresponding to the session identifier cached in the prepared statement cache library is a prepared statement session where a connection is successfully established between a client and a database.