CDC Processing Mode Switching for Database Log Management

Solution Overview

Problem

Current Change Data Capture (CDC) technologies face challenges in efficiently processing redo logs due to resource utilization imbalances and storage limitations, leading to potential data loss and increased resource usage, especially when extracting data from archive redo logs.

Innovation Solution

A computer program that determines the processing mode for change data based on remain log and resource utilization rate information, switching between saving data as a CDC file or recording it in memory to optimize resource usage and prevent data loss.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If change data is extracted from archive redo logs, then data completeness is improved, but resource utilization increases and storage capacity is exceeded

Engineering Contradiction:

Improvedata completenessVSAvoidresource utilization

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent implements dynamic mode switching between first mode (saving as CDC file) and second mode (recording in memory) based on real-time remain log information and resource utilization rate. This allows the system to adaptively adjust its operation to balance data completeness with resource consumption, extracting data from archive redo logs when resources permit while preventing resource exhaustion.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes operational parameters by switching between two distinct processing modes. In the first mode, change data is saved as CDC files with lower resource utilization. In the second mode, change data is recorded in memory with higher resource utilization. The mode selection is determined by comparing remain log information and resource utilization rate against thresholds, enabling flexible parameter adjustment to resolve the contradiction.

Inventive Principle:

Principle #35Parameter changes

2Quantity of substance

If more redo log files are used to store accumulated redo logs, then storage capacity is improved, but device complexity increases

Engineering Contradiction:

Improvestorage capacityVSAvoiddevice complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent divides redo log storage into multiple redo log files (first redo log file, second redo log file, etc.) and processes them in a segmented manner. The system extracts change data from archive redo logs while managing multiple active redo log files, allowing storage capacity to expand without proportionally increasing complexity through structured segmentation of log management.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system implements a循环利用 mechanism where redo log files are reused after their data has been processed. When one redo log file is filled, the system switches to another file, and previously filled files can be reused after their contents have been extracted. This reduces the need for continuously increasing storage capacity while maintaining manageable complexity.

Inventive Principle:

Principle #34Discarding and recovering

3Speed

If change data is recorded in memory, then processing speed is improved, but resource utilization rate increases

Engineering Contradiction:

Improveprocessing speedVSAvoidresource utilization rate

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The patent implements dynamic mode switching between first mode (saving as CDC file) and second mode (recording in memory) based on real-time remain log information and resource utilization rate. This allows the system to adaptively adjust its operation to balance data completeness with resource consumption, extracting data from archive redo logs when resources permit while preventing resource exhaustion.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes operational parameters by switching between two distinct processing modes. In the first mode, change data is saved as CDC files with lower resource utilization. In the second mode, change data is recorded in memory with higher resource utilization. The mode selection is determined by comparing remain log information and resource utilization rate against thresholds, enabling flexible parameter adjustment to resolve the contradiction.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11726978B2Computer program for providing efficient change data capture in a database system

Publication Date: 2023.08.15 SILCROAD SOFT INC

AI summary

An exemplary embodiment of the present disclosure discloses a computer program stored a non-transitory computer readable storage medium. When the computer program is executed by one or more processors, the computer program performs operations for Change Data Capture (CDC) by one or more processors, and the operations include: extracting a change data from at least one redo log of an online redo log and an archive redo log; analyzing at least one information of a resource utilization rate information and a remain log information of a source database server; and determining to process the change data in one mode of a first mode saving the change data as a CDC file and a second mode recording the change data in a memory, based on at least one of the remain log information and the resource utilization rate information.