Nested Data Sample for Consumer Behavior Analysis

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The challenge lies in integrating and analyzing diverse data sets from various sources to gain commercially relevant insights, as existing methods face difficulties in handling structured and unstructured data, proprietary nature of data sets, and the sheer volume of data, which makes direct analysis infeasible.

Innovation Solution

A computerized method that identifies a universal data set, partitions a geographic area into sectors, extracts data from ancillary sources, and combines it with panel data, using techniques like propensity modeling and shrinkage estimators to create a nested data sample that allows for inference and analysis of consumer behaviors across sectors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If data is stored at various locations and in various forms from multiple sources, then the quantity and diversity of available data increases, but the difficulty of integrating and analyzing the data increases

Engineering Contradiction:

Improvequantity and diversity of dataVSAvoiddifficulty of integrating and analyzing data

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The geographic area is partitioned into multiple sectors, and data is extracted from ancillary sources in an order consistent with enumerated sectors. This segmentation allows the system to handle large volumes of diverse data by processing it in manageable, organized units rather than attempting to integrate all data simultaneously.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A data integration system acts as an intermediary between multiple ancillary data sources and the analysis platform. This mediator extracts data from various sources (point of sale, shipping, media delivery, credit card, clickstream data) and combines it with panel data, managing the complexity of integration while preserving the diversity of data sources.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If the geographic area is partitioned into many sectors to enable detailed analysis, then the precision of consumer behavior analysis improves, but the time and computational resources required increase

Engineering Contradiction:

Improveprecision of consumer behavior analysisVSAvoidtime and computational resources required

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

Sectors are enumerated in advance before data extraction begins. This preliminary organization of the geographic partitioning structure allows the system to efficiently navigate and extract data from ancillary sources without repeatedly calculating sector boundaries during the extraction process, reducing computational overhead.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system extracts data from ancillary sources until meeting data thresholds for each sector, rather than exhaustively collecting all possible data. This partial action approach achieves sufficient precision for consumer behavior analysis while limiting the time and computational resources required by stopping extraction once adequate data is obtained.

Inventive Principle:

Principle #16Partial or excessive action

3Quantity of substance

If data is extracted from multiple ancillary sources to comprehensive data collection, then the completeness of consumer behavior data improves, but the complexity of data extraction and combination increases

Engineering Contradiction:

Improvecompleteness of consumer behavior dataVSAvoidcomplexity of data extraction and combination

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The data extraction system is designed to handle multiple types of ancillary data sources (point of sale data, shipping data, media delivery data, credit card data, clickstream data) through a unified extraction process. This universal approach extracts data from diverse sources using consistent methods, improving completeness while managing complexity through standardization.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

Data extraction from ancillary sources continues in an ordered sequence through enumerated sectors until data thresholds are met for each sector. This continuous extraction process ensures comprehensive data collection across all sources while maintaining systematic control over the complexity of the operation.

Inventive Principle:

Principle #20Continuity of useful action

4Adaptability or versatility

If propensity modeling is performed for sectors without sufficient data, then the ability to analyze all sectors improves, but the accuracy of propensity estimates decreases

Engineering Contradiction:

Improveability to analyze all sectorsVSAvoidaccuracy of propensity estimates

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The system checks whether sufficient data is available for each sector before performing propensity modeling. When data thresholds are not met, the system identifies these sectors and can apply appropriate handling methods, using feedback from data availability assessment to guide the modeling process and maintain accuracy standards.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS8781877B2Data integration and analysis

Publication Date: 2014.07.15 INFORMATION RESOURCES INC

AI summary

Uncorrelated data from a variety of sources, such as consumer panels or retailer points of sale, are combined with maximal coverage of a universal data set for a population in a manner that permits useful inferences about behavioral propensities for the population at an individual or household level.