Distributed Financial Data Aggregation via Serialized Binary Schemas
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for consolidating risk and profit/loss information in commercial investment banks face complexity due to large datasets, leading to conflicts between optimal data storage and computation forms, and increased development and deployment costs.
Innovation Solution
An automated computer system utilizing a relational database with staging, recent, and historic schemas, coupled with a distributed data storage platform, processes financial data by creating aggregation specifications, composing keys, and combining aggregations to create new algorithms, allowing for efficient aggregation and disaggregation of financial information without the need for deserialization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in a centralized form for long-term storage, then storage efficiency is improved, but computation speed and aggregation performance deteriorate
Solution Approach 1:
The patent divides the centralized data storage system into multiple distributed nodes, each storing portions of the financial data. This segmentation allows parallel processing of aggregation algorithms across nodes, improving computation speed while maintaining storage efficiency through distributed architecture.
Solution Approach 2:
The patent introduces a new dimension to data storage by creating serialized binary forms that can be stored efficiently while enabling fast computation. This dimensional transformation from traditional row/column storage to a serialized format optimized for both storage and computation resolves the contradiction between storage efficiency and aggregation performance.
2Measurement precision
If complex aggregation algorithms are developed to handle large datasets, then measurement precision is improved, but algorithm complexity and development cost increase
Solution Approach 1:
The patent changes the parameters of data representation by serializing financial data into binary forms with specific schemas. This parameter transformation enables simpler aggregation algorithms to achieve the same measurement precision as complex algorithms, reducing development complexity while maintaining accuracy.
Solution Approach 2:
The patent creates serialized copies of financial data in optimized binary formats that preserve all necessary information for accurate aggregation. These copied representations enable precise risk and profit/loss measurements while allowing use of efficient, simpler aggregation algorithms rather than complex processing of original data formats.
3Quantity of substance
If data is transformed to optimal storage form, then storage efficiency is improved, but computation flexibility deteriorates
Solution Approach 1:
The patent creates a universal serialized data format that serves multiple functions: efficient long-term storage, fast aggregation computation, and flexible querying. This multi-functional data representation eliminates the need to transform data between storage and computation forms, maintaining both storage efficiency and computation flexibility simultaneously.
Data Source
AI summary
According to an embodiment of the present invention, a system and method for consolidating financial data comprising: a relational database containing a plurality of schema comprising a staging schema, recent schema and historic schema; a distributed data storage platform comprising a plurality of nodes; and a computer processor, coupled to the relational database and the distributed data storage platform, and programmed to: store financial data in the distributed data storage platform comprising the plurality of nodes; create an aggregation specification to compute an aggregation for a financial measure; determine whether the aggregation is current; compose one or more keys for the aggregation specification; determine one or more dimensions for the keys; responsive to the keys and dimensions, process the aggregation specification via the plurality of nodes; and combine one or more aggregations to create a new aggregation algorithm.