Process Control Tool for Big-and-Wide Manufacturing Data Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack the capability to efficiently perform real-time or near real-time analytical challenges associated with Big-and-Wide data in complex manufacturing scenarios, particularly in semiconductor manufacturing, where high-dimensional data with discrete variables and millions of parameters pose challenges for parallelization and data processing.
Innovation Solution
A distributed processing system comprising a multi-node parallel data processing system for initial feature selection and data transformation, followed by in-memory computations on a dedicated analytics server, utilizing techniques like impact-coding and SAX-style coding to reduce data dimensionality and enable efficient processing of wide and big data structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If distributed parallel processing is used for Big-Data with many rows, then computational performance is improved, but the system cannot efficiently handle Wide-Data with millions of columns
Solution Approach 1:
The patent segments the wide data processing into two distinct phases: (1) an in-database parallel phase that performs initial feature selection and data transformation on distributed nodes, and (2) an in-memory sequential phase that performs final analysis on a dedicated analytics server. This segmentation allows the system to leverage parallel processing for data preparation while using sequential processing for the computationally intensive final analysis, thereby resolving the contradiction between parallel performance and wide-data handling capability.
Solution Approach 2:
The patent introduces an intermediary data transformation layer that converts wide data formats into stacked formats suitable for parallel processing. This intermediary transformation enables the system to accept wide data inputs, transform them into a parallel-friendly format, process them distributively, and then convert results back to the required output format, thereby bridging the gap between parallel processing capabilities and wide-data requirements.
2Speed
If all data is processed in-memory on a dedicated server, then analysis speed is improved, but memory requirements and computational overhead become prohibitive
Solution Approach 1:
The patent applies preliminary action by performing feature selection and data transformation in the first phase before the final analysis. By pre-processing the data to identify and retain only the most relevant features and transforming it into an appropriate format, the system reduces the volume and complexity of data that needs to be loaded into memory for the final analysis, thereby improving analysis speed while reducing memory requirements and computational overhead.
3Productivity
If traditional analytics push computations to parallel nodes, then row-based processing is efficient, but column-based feature selection becomes inefficient
Solution Approach 1:
The patent segments the feature selection process into column-based operations performed during the in-database parallel phase, where statistics are computed for each column independently across distributed nodes. This segmentation allows efficient column-based feature selection to occur in parallel, while maintaining the row-based processing efficiency for the main data flow, thereby resolving the contradiction between row processing efficiency and column-based feature selection efficiency.
Data Source
AI summary
A process control tool for processing wide data from automated manufacturing operations. The tool including a feature selector, an analysis server, and a visualization engine. The feature selector receives process input data from at least one manufacturing process application, wherein the process input data includes a plurality of observations and associated variables, converts the received process input data to a stacked format having one row for each variable in each observation, converts identified categorical variables into numerical variables and identified time-series data into fixed numbers of intervals, computes statistics that measure the strengths of relationships between predictor values and an outcome variable, orders, filters, and pivots the predictor values. The analysis server performs at least one operation to identify interactions between predictor values, e.g. using maximum Likelihood computations or predefined searches, in the filtered predictor values. The visualization engine displays the interactions for use in managing the manufacturing operations.