TDS Row Bitmap Encoding for Null Column Compression

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current Tabular Data Stream (TDS) protocols are inefficient in transmitting query results due to the transmission of null columns, which consume bandwidth without providing useful data.

Innovation Solution

Incorporating an identification component that uses a bit map to differentiate null and non-null columns, allowing only non-null columns to be sent, thereby reducing the number of bits transmitted and improving efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of energy

If all columns including null columns are transmitted in TDS protocol, then complete data structure is maintained, but bandwidth is wasted and transmission efficiency deteriorates

Engineering Contradiction:

Improvebandwidth usageVSAvoidtransmission efficiency

Core Design Contradiction:

Loss of energyVSProductivity

Solution Approach 1:

The patent extracts and removes null column data from the transmission stream, sending only non-null column values. This is achieved by identifying null columns and excluding them from the data transmission, thereby reducing bandwidth consumption while maintaining the integrity of actual data.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent segments the data transmission into two parts: a bitmap indicating which columns are null, and the actual non-null column data. This segmentation allows the receiver to reconstruct the complete data structure by placing null values in appropriate positions based on the bitmap information.

Inventive Principle:

Principle #1Segmentation

2Productivity

If null columns are excluded from transmission, then bandwidth is saved, but data structure completeness may be compromised

Engineering Contradiction:

Improvetransmission efficiencyVSAvoiddata structure completeness

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The patent introduces a bitmap as an intermediary data structure that carries metadata about column nullness. This bitmap acts as a guide for the receiving end to reconstruct the complete data structure, ensuring that even though null columns are not transmitted, their positions and types are preserved through the bitmap information.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If bitmap is added to indicate null columns, then transmission efficiency improves, but header size increases

Engineering Contradiction:

Improvetransmission efficiencyVSAvoidheader structure

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent changes the representation parameters of column data by using a compact bitmap format instead of transmitting full column definitions for null columns. This parameter change reduces the overall transmission size despite adding a header, because the bitmap uses minimal space (one bit per column) compared to the space that would be required to transmit actual null column data.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentEP2208317B1Compressing null columns in rows of the tabular data stream protocol

Publication Date: 2019.09.25 MICROSOFT TECHNOLOGY LICENSING LLC

Figure 1
Figure 2
Figure 3

AI summary

Systems and methodologies that enhance a Tabular Data Stream (TDS) protocol by enabling efficient transmission of a row(s) with null columns(s). An identification component employs a bit map that can be positioned at beginning of a row, to indicate to the receiving side (e.g., a client) columns that are to be sent. Accordingly, by distinguishing columns that are null from columns that are not null - followed by sending columns that are not null - transmission resources can be effectively employed.