Localized Free Space Maps for Scalable Distributed Databases

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The shared-everything architecture in multi-datanode databases faces challenges with contention and increased execution times due to page negotiations and serialization during free space management, leading to inefficiencies and reduced scalability.

Innovation Solution

Implementing localized Free Space Maps (FSMs) for each datanode, allowing each node to manage its own FSMs, and using a dynamic FSM assignment and recycling algorithm to minimize cross-node negotiations and ensure efficient free space management.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a shared-everything architecture is used with a shared FSM, then any data node can access any file within the database, but contention for free space management increases execution times due to page negotiations

Engineering Contradiction:

Improveaccess flexibilityVSAvoidexecution time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The shared Free Space Map is segmented into multiple localized FSMs, with each datanode having its own dedicated FSM for managing free space. This segmentation eliminates contention by allowing each node to independently manage its own free space without negotiating with other nodes, thus reducing execution time while maintaining distributed access capability

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Each datanode is assigned a localized FSM that is specific to its own free space management needs. This local quality approach allows each node to have customized free space tracking without interfering with other nodes, resolving the contradiction between shared access and contention reduction

Inventive Principle:

Principle #3Local quality

2Reliability

If multiple datanodes share a single FSM, then free space can be centrally managed, but the system lacks scalability when adding new datanodes

Engineering Contradiction:

Improvefree space management consistencyVSAvoidscalability

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The system dynamically assigns localized FSMs to datanodes based on their specific needs and workload. When new datanodes are added, they automatically receive their own FSM instances, enabling the system to scale dynamically without requiring reconfiguration of a shared FSM, thus maintaining both consistency and scalability

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

Each localized FSM implements the universal free space management functionality independently. Multiple FSMs provide the same core functions as a single shared FSM would, but in a distributed manner that supports system expansion and maintains management consistency across all nodes

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12619577B2Method and apparatus for free space management

Publication Date: 2026.05.05 HUAWEI TECH CO LTD

AI summary

There is provided a method and apparatus for free space management in a distributed database. Each datanode of the database is assigned a free space map to manage free space. When a datanode requires additional storage space, a new free space map may be assigned to the datanode, a free space map of a different datanode may be shared with the datanode, or a free space map of a different datanode may be reassigned to the datanode. A background process may be used to recycle inactive free space maps.